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The Validation of an Automated Counseling System 

by 

Deloss David Friesen* 

ABSTRACT 

SDC document TM-2582, entitled "Explorations in Computer -Assisted Counseling 
by John F. Cogswell and Donald T. Estavan, describes research that eventuated in 
a computer-based model that was dssigned to simulate the behavior of a school 
counselor in two related tasks; the appraisal of students from data in the 
cumulative student record and the educational planning interview. 

Three institutions cooperated in the research project --System Development 
Corporation, University of Oregon, and the Palo Alto School District. 

Dr. Friesen 's paper, which represents his doctoral dissertation, is con- 
cerned with the validity of the automated systems. Forty 9th-grade students 
from the Palo Alto School District participated in the validation study. The 
cumulative records of the 40 students were analyzed by the automated appraisal 
program and the 40 students participated in the automated educational planning 
interview. Twenty of the 40 students also were interviewed by the original 
counselor whose behavior was being simulated by the automated program. The 
other 20 students were interviewed by a second counselor from the school district 
Both counselors made appraisals of the students from the student cumulative 
record. The performance of the autcmiated counseling system was compared to the 
performance of the two counselors ori the following four variables; the appraisal 
of pupil records prior to the interview, the post -interview appraisal of students 
the educational decisions made by the students, and the completeness of students' 
educational plans . 



^Formerly of the Graduate School in Education at the University of Oregon. 
Currently with the Counseling Center, State University of New York, Albany, 
New York. 
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CHAPTER 1 

STATEMENT OF THE PROBLEM 
Introduction 

A school counselor of today is faced \vith serving approximately 
four hundred^ students each year. A professional counselor's duties 
and obligations are varied. In addition to counseling with students 
concerning educational, vocational, and personal problems, he may 
work as a registrar in developing class schedules and assigning students 
to these classes, keep attendance records, check on truants, write 
progress reports, consult with staff members, discuss problems with 
parents, and dispense educational, vocational, and other guidance infor- 
mation. Thus, most counselors have a multitude of activities to 
perform for a large and varied group of people. However, some of 
these tasks are somewhat routine in the sense that, while complex, 
they involve generally repetitive aspects. 

Repetitive tasks generally involve a much smaller number of 
variables than tasks which deal with unique student problems. Because 

^C. Gilbert Wrenn, The Counselor in a Changing World , American 
Personnel and Guidance Association, 1962, p. 192. 
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of the relatively small number of variables in some tasks, it should 
be possible to isolate and identify the component procedures necessary 
to complete the task. If all relevant components could in fact be 
identified, then it would be possible to write a computer program 
which, when put into operation, would yield results similar to those 
obtained by a counselor. The automation of selected counseling tasks 
would have several implications: (1) it would permit a greater volume 
of work to be handled at a faster pace -- thus initiating a financial 
saving for a school district; (2) it wouldincreasethe volume of information 
stored, decrease the retrieval time of stored information, and permit 
a more thorough and sensitive analysis of information now available in 
schools; and (3) itwould allow the counselor to spend more time in 
counseling with students needing help with non- repetitive problems. 

Problem 

The problem with which this study is concerned is the validity 
of a computer-based counseling system, which has been developed to 
simulate the decision-making behavior and logic of a counselor 
counseling with ninth grade students in regard to their educational 
planning for high school. 



■ » 
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Background and Rationale for the Study 

While this study is not concerned with the development o£ the 
automated counseling system per se, it may be helpful to review the 
background and rationale of the automated counseling system to under- 
stand more clearly the nature of the automated system and the problem 
it seeks to alleviate. 

Demand for Counseling Increasing 

The demand for counseling services in schools is increasing 
rapidly. Wrenn^ indicates that the number of high school age people, 
14-17 years of age, will double in the period between 1950 and 1980. 

The increased complexity of decisions created by our rapidly develop- 
ing society will cause the job of the school counselor to become more 
important and demanding. 

Two Basic Alternatives for Meeting Demands 

The Addition of More Personnel. 

The Use of Automated Techniques in Counseling. 

This would have the following results: 

1 . automating repetitive tasks, and 

2. increasing the use of information. 

^Ibid. , p. 17. 
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Rationale for the Selection of the 
S econd Alternative 

The first alternative is unlikely because the addition of more 
personnel is expensive, and a decrease in the pupil- counselor ratio 
is not probable. Although more counselors are being prepared, the 
growth of the schools is keeping the pupil- counselor ratio about the 
same. Since there does not appear to be, in the near future at least, 
a decrease in the number of students per counselor, the necessity of 
relieving a counselor of many routine duties so that he can spend more 
time with students in group or individual counseling sessions is clear. 
Even if the pupil- counselor ratio were lowered, the addition of personnel 
would not materially effect the need for better storage techniques, 
more rapid information retrieval, analysis, and the constant updating 
of information. 

Not only would the automation of certain counseling tasks be 
beneficial but it is also technically possible. Computer specialists 
and systems people, as discussed at the 1964 Western Computer 
Conference, feel that any activity or process that can be defined in 
operational terms can be automated. 

Other applications of computer techniques indicate that auto- 
mation of specific tasks is possible. However, questions concerning 
validity and reliability are raised when other computer research is 
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applied to a counseling system. It was necessary to investigate the 
more specific applicability of computer technology to specific counsel- 
ing tasks. 

Before the theory of anything that can be operationally defined 
can be automated could be put into practice, it was necessary not only 
to find a task but also to find a counselor whose behavior could be 
defined. 

Selection of the Tas k 
3 

Tyler says that much of counseling is oriented toward the 

4 5 

making of decisions. Loughary and Gelatt also f«..*el that decision- 
making is one of the most important functions of the adolescent, and 
many of the problems of high school students arise from the inability 
to make appropriate decisions. 

Working from a decision-making frame of reference, then, it 
was necessary to select a decision-making task to be simulated. 

3 

Leona E. Tyler, The Work of the Counselor (New York: 
Appleton- Century- Crofts, Inc., 1953), p. 228. 

4 

John W. Loughary, Counseling in Secondary Schools (New York; 
Harper & Brothers, 1961), p. 23. 

5 

Harry B. Gelatt, "Decision Making: A Conceptual Frame of 
Reference for Counseling, " The Journal of Counseling Psychology, 

Vol. 9, No. 3, 1962, pp. 240-246. 
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The educational planning interview of ninth grade students in 
preparation for high school was chosen because it is a decision-making 
task, it represents a considerable amount of time which must be 
expended by a counselor, the number of alternatives involved are 
fewer than would be found in decision-making tasks of a more complex 
nature, and high school counselors have defined educational planning 
interviews as repetitive tasks. 

Selection of the Counselor 

In the preliminary study the counseling behavior of over forty 
secondary school counselors from various parts of the United States, 
who were selected by their supervisors as being expert counselors, 
was analyzed. This analysis was to determine if their counseling 
techniques, while performing a specific counseling task, could be 
operationally defined to permit possible future programing for use in 
a computer-based interview. 

This research was done to determine how much, and what 
aspects, of counselor pre- interview (verbalized thoughts about the 
student and his past, present, and future status as gained through 
reading the student's cumulative record folder) and interview (verbal 
interchange between the counselor and student during the counseling 
interview) behavior can be automated, i. e. , performed by a computer 
and related equipment. 



o 
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The results of the preliminary study indicate that many high 
school counselors behave, while performing counseling functions, 
according to more or less systematic sets of rulesr These rules 
varied from the making of predictions a.bout the situdent based on family, 
social, and personal factors to more concrete rules such as academic 
predictions based on GPA, telling students they must have a B average 
to carry five solids, or how many special units a student must have 
for entrance into a certain type of university. These rules can be 
identified by an analysis of counselors' tape-recorded behaviors, 
interview and pre- interview, using a systems approach, and with 
de-briefing sessions in which the counselors were asked to clarify 
their logic and behavior. These rules can be programed so that a 
computer, with the aid of an input- output device -- similar in 
appearance to a typewriter -- could interview the student. 

Although several counselors were found who met the criteria 
of using consistent rules in both interviewing and pre- interviewing 
functions, only one was chosen to be the model for the automated 
counseling system. Not only did he use consistent rules which could 
be readily identified, but the school district in which he works has 
done an extensive follow-up on the students which allows the counselors 
to make accurate predictive statements about the students. 
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The Automated Counseling System 

Based on the findings of the preliminary study, a research team 
developed a computer-based system which attempts to simulate the 
counseling behavior and logic exhibited by one secondary school 
counselor during educational planning interviews with ninth grade 
students. 

In addition to deter ming how well the automated counseling 
system simulates the outcomes of the model counselor, the extent of 
agreement between the automated system and a second counselor is 
investigated. If an automated counseling system is to have practical 
value, it must not only produce results similar in nature to those of 
the model counselor, but should also produce results similar to those 
of other counselors in the school district who are using the same data 
and norms used by the model counselor. 

The Purpose of this Study 

The present study attempts to ascertain whether or not the 
simulated counseling program actually duplicates the results achieved 

^The research team was composed of Dr. J. W. Loughary, 
Robert Hurst and the writer -- doctoral students at the University of 
Oregon -- and Dr. John Cogswell and Donald Estavan of Systems 
Development Corporation. 



o 
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by the model counselor and the second counselor in their usual 
counseling interviews. 

Implications 

The introduction of an automated counseling system into a 
school system would offer some advantages in addition to educational 
planning. The automated counseling system being investigated in this 
study would need minor modifications in some cases to be suited for 
the projected tasks listed below. 

Group Counseling. While having an interview with the automated 
system a student could request a personal interview with the counselor. 
After making his request the student could also indicate if a group 
counseling session would be satisfactory. If so, he would state the 
nature of his problem. Many of the students' problems may be 
similar in nature. Instead of having, for example, twenty-five 
individual counseling sessions, on the basis of these self- referrals, 
the counselor may find it beneficial to have three or four group sessions, 
each taking a particular problem area. For those students still having 
problems unique to themselves, individual counseling sessions could 
be arranged. 

Scheduling of Interviews. After investigating the interviews 
of students who have been counseled by the automated system, it may 
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be possible to isolate four or five problem areas most frequently used 
as reasons for requesting further counseling. These problem 
categories could be included, along with another category labeled 
"other problems, " in a revised machine interview program. If the 
computer were given the time schedules of the students and the 
counselor, on the basis of the problem area chosen in the automated 
interview, both group and individual counseling sessions could be 
scheduled by the computer. A print- out giving the time and the place 
of the interview could be given to the counselor and the students. 

Collect, Store, and Retrieve Information. As more schools 
are initiating research projects the collection of data becomes more 
important. The more often these data are used the more often they 
must be up-dated. In the up-dating process and other applications 
the speed of retrieval becomes highly important. As school systems 
grow and as more areas of importance about people are found th€i 
sheer bulk of information is tremendous. 

One of the side benefits of having an automated counseling 
system is that the equipment necessary, electronic data processing 
equipment, for the handling of this vast quantity of information would 
be in operation — needing only further programing to extend its range 



of usefulness. 
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Problems for Further Research in 
Counselor Simulation 

Pre- college Educational Planning . The transference between 
planning for high school and for college would be relatively easy. The 
logic would be the same but the complexity involved in college planning 
would be much greater. Instead of meeting the requirements for one 
high school, the computer would have to have courses, prerequisites, 
and requirements for a large number of colleges and universities stored 
in its memory. 

Vocational Planning. Vocational planning is related to the 
educational interview because of the linkage between most occupations 
and the training necessary for these occupations. A great deal of 
information concerning training requirements, entry occupations, 
salary, job descriptions, and working environment would have to be 
stored in the computer. Although the task is large the new large 
core-memory computers make it possible. 

Personal Problem Counseling. This is the area where there 
would probably be the greatest number of situations and problems 
unique to the individual. These problems are usually not resolved in 
just one interview. As unique as these problems are, there may be 
found similarities which could be programmed. The automated 
interview system in personal problem counseling could be used as the 
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first, or intake, interview where general questions and problems are 
handled. The succeeding interviews wou i be more specific, and 
handled by a counselor who was aware of the problems expressed during 
the machine session. The computer could also schedule the student 
for group sessions with other individuals expressing the same problems. 

Developing a Counselor Model . Instead of simulating the 
behavior of one counselor, research might be generated in the direction 
of looking at a group of expert counselors and simulating the total 
group's behavior and thinking. The strengths of one counselor might 
then be used to overcome the weaknesses of another. Any of the 
previously mentioned counseling tasks could be used in this group 
simulation project. 



Objectives 



General Objective 

The general objective of the research with which this study is 
concerned is to determine the validity of an automated counseling 
system. The validity criteria, which are discussed more fully below, 
are the outcomes of the counseling interview after which the automated 



counseling system was modeled. 
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# 



Specific Objectives 



The proposed study has the four following specific objectives: 

1, To determine and demonstrate the extent to which a 
computer-based counseling system can produce the same results as 
the traditional counselor, after whom the computer-based system was 
modeled. The extent of agreement is determined by the amount 

of difference between the two interview modes in regard to: 

a. pre- interview pupil appraisal. On the basis of 
cumulative record data, both counseling modes predicted the 
high school GPA. Fourteen other appraisal statements relating 
to such factors as underachievement, overachievement, and 
probability of future academic difficulties were generated, 

b. post- interview pupil appraisal. The realism of the 
student's plans and courses selected was determined by each 
counseling mode, 

c. educational decisions -- for each interviewing mode 
a course schedule fo r the high school years was made, and 

d. completeness of educational plans after each inter- 
view the number of students making total plans, partial plans, 
and no plans, vas recorded. 

2. To determine the extent of agreement between the automated 
system and a second counselor from the same school district as the 
model counselor, but who was not involved in the rule-building phase 
of the study. The second counselor is compared with the automated 

counseling system in regard to: 

a. pre-intetview pupil appraisal, 

b. post- interview pupil appraisal, 

c. educational decisions, and 

d. completeness of educational plans. 
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3. To compare the differences between the students interviewed by 
the automated counseling system and the model counselor, and the 
students interviewed by the automated counseling system and the 
second counselor in terms of interview outcomes in regard to: 

a. pre- interview pupil appraisal, 

b. post- interview pupil appraisal, 

c. educational decisions, and 

d. completeness of educational plans. 

4. iO identify specific limitations of the computer-based 
counseling system. 

a. machine limitations -- are the rules and the information 
on which the program was built general and inclusive enough to 
allow the student to select a course program comparable with 
one selected by being interviewed by a counselor? If the student 
is unable to make total course plans, does he see this as related 
to the clarity of the presentation or to the amount of information 

of the automated interview ? 

b. attention span -- does the student respond to the 
automated counseling system by early ternunation of the 
interview, and by stating that the machine interview made him 
bored and restless ? 

c. lack of reinforcers --do the students feel that lack 

of feedback, knowing whether their response was appropriate or 
inappropriate, made it more difficult to select courses ? Do the 
students feel that this lack of feedback was because of the design 
of the machine program or the lack of the presence of a counselor? 

d. lack of sensitivity to the student's needs and orientation 
towards counseling -- does the student feel that the automated 
interview meets his need in regard to high school course 
planning? Although the machine is not a human being, does it 
meet the student's expectations of what a counseling session in 

educational planning should be ? 

e. lack of student cooperation -- what percentage of the 

students attempt to jam the machine or input ridicxilous 
responses? 



20 August 1965 



25 



TM- 26 11/000/00 



f. reservations about course plans -- how does the group 
counseled by the machine compare with those counseled by a 
counselor in terms of certainty about chosen courses ? 

Hypotheses 

Regarding the specific objectives the following three major 
hypotheses are tested. 

1. The automated counseling system will not be significantly 
different from the model counselor regarding: 

a. pre- interview pupil appraisal, 

b. post- interview pupil appraisal, 

c. educational decisions, and 

d. completeness of educational plans. 

2. The automated counseling system will not be significantly 
different from the second counselor regarding: 

a. pre- interview pupil appraisal, 

b. post- interview pupil appraisal, 

c. educational decisions, and 

d. completeness of educational plans. 

3. The extent of agreement between the automated system and 
the model counselor will not be significantly different from the extent 
of agreement between the automated system and the second counselor 
regarding: 



lip 
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a. prc- interview pupil appraisal, 

b. post- interview pupil appraisal, 

c. educational decisions, and 

d. completeness of educational plans. 

Summary 

This chapter has discussed the background and rationale of the 
study, pointing up the need for such an investigation in the light of 
the demands of the job now being placed on counselors and the 
increasing demands of the job in the future. Also, it was indicated 
that such a study is technically possible, and some of the implications 
for future applications of computer technology were given. It has 
stated that the problem is the validity of a computer-based counseling 
system, which was developed by attempting to simulate the outcomes 
produced by a secondary school counselor, and the general objective 
is to determine the validity of this automated system. The specific 
objectives involve: (1) determining the agreement between the auto- 
mated system and the model counselor in regard to pre- interview pupil 
appraisal, post interview pupil appraisal, educational decisions and 
completeness of educational plans; (2) determining the agreement 
between the automated system and the second counselor in regard to 
pre- interview pupil appraisal, post interview pupil appraisal. 
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educational decision, and completeness of educational plans; 

(3) comparing the differences of those students counseled by the 
machine and the second counselor regarding pre- interview pupil 
appraisal, post interview pupil appraisal, educational decisions, and 
completeness of educational plans; and (4) identifying specific limita- 
tions of the computer-based counseling system in regard to machine 
limitations, attention span, lack of reinforcers, lack of sensitivity, 
lack of student cooperation, and reservations about course plans. 

In terms of the specific objectives the following three major hypotheses 
are tested: (1) the automated counseling system will not be significantly 
different from the model counselor regarding pre- interview pupil 
appraisal, post- interview pupil appraisal, educational decisions, and 
completeness of educational plans; (2) the automated counseling system 
will not be significantly different from the second counselor regarding 
pre- interview pupil appraisal, post- interview pupil appraisal, 
educational decisions, and completeness of educational plans; and 
(3) the extent of agreement between the automated system and the 
model counselor will not be significantly different from the extent 
of agreement between the automated system and the second counselor 
regarding pre- interview pupil appraisal, post- interview pupil appraisal, 
educational decisions, and completeness of course plans. 
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CHAPTER II 

REVIEW OF THE LITERATURE 

This chapter will attempt to support the importance and the 
feasibility of this study in two areas: (1) the need for counseling 

services; and (2) the technical possibility of constructing an automated 
counseling system. The first area will be investigated by identifying 
the changes in our society which influence the need for greater 
counseling services. The second area will be concerned with reviewing 
theoretical and research literature in human simulation and decision- 
making by electro^iic computers. 

Need for Counseling Services 
Increase in Population 

On January first of 1964 the population of the United States 
reached almost 191 million people. The annual growth in millions is 
currently about 2. 7 (1.3%). Eighty-nine percent of the growth is due 
to net population increases -- births minus deaths -- and 11% is from 
civilian migration (Current Population Reports, Series P-25, No. 278, 
January 29, 1964). ^ 

^MaxF. Baer, ’’Washington Flashes.'* The Personnel and 
Guidance Journal, Vol. XLII, No. 8 (April, 1964), p. 744. 
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Youth Unemployment 

The youth unemployment figure for whites has risen from 17. 7% 
in 1962 to 20. 5% in 1964 for the 14- 19 year olds, and from 8. 0% to 
8. 5% for the 20-24 year olds. Non- whites had an unemployment rate 
of 22. 5% in 1962, which has risen to 33. 2% in 1964 for the 14-19 year 
olds and has dropped from 15.8% to 15% for the 20-24 year olds. The 
spread between the whites and the non-whites has increased for the 

2 

14-19 age bracket but has dropped slightly for the 20-24 year olds. 

With a combined school drop-out rate of 620, 000 out of 5. 2 million 

3 

(11.9%), the problem does not appear to be diminishing. 

Juvenile Delinquency 

In 1962 four hundred and seventy thousand youthful offenders 
came before the courts. This figure represents 1.8% of all children 
aged 10-17 in the country. The population increase for 1962 was 3. 5%, 

while the increase in juvenile delinquency was 10% (Children's Bureau 

4 

Statistical Series 73, Juvenile Court Statistics -- 1962). 

^Max F. Baer, "Washington Flashes, " The Personnel and 
Guidance Journal, Vol. XLIII, No. 2 (October, 1964), p. 112. 

^Max F. Baer, "Washington Flashes, " The Personnel and 
Guidance Journal, Vol. XLIII, No. 1 (September, 1964), p. 4. 

4 

Ibid. 
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Increase in School Enrollment 

During the years of 1961-63 ninety- one percent of the students 

aged 16 attended, as contrasted to 88% for 1957-59. Seventy- eight 

percent of the 17 year olds were in school during 1961-63 compared to 

65% in the period of 1957-59. For the cclleges during the period of 

1958-63 enrollment increased by 33. 7% (Current Population Survey 

5 

conducted by Bureau of the Census). 

Educational Changes 

A comparison of educational mobility during a generation 
involving men ZO-64 years of age indicated that 55% of men were high 
school graduates compared with 24% of their fathers. About 26% of 
the men had completed one or more years of college, while only 10% 
of their fathers attended college. Also, 39% of their fathers lacked an 
eighth grade education compared with only 14% of the sons. Among the 
father s who did not complete the eighth grade, 75% of iheir sons 
completed the eighth grade, and 37% completed high school. Forty- 
one percent of the sons who had fathers who graduated from high school 
but did not attend college had at least some college. Also, of the 
fathers who attended but did not graduate from college, 41% had 

^Ibid. 
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sons who did graduate (Education of the American Population by Folger 
and Man). ^ 

Need for Decision-Making 

Many of the problems experienced by adolescents are created by 
the inability to make appropriate decisions. The student who continually 
creates a classroom disturbance, the juvenile engaging in delinquent 
acts, the youngster who fights with his parents, or the student who is 
distressed about his place in the world have goals towards which they 
are striving. It is not the goal that is creating the problem but the 
inability to identify alternate courses and the selection of the appropriate 

plan of action for meeting the goal. 

One of the major functions of guidance is to aid students to 
discover alternative plans of action for their goals and to select the 
one most appropriate to them. Rothney states that "guidance is largely 
concerned with choices that individuals make among present and 
future actions. " 

Effective decision-making is a skill that is learned. Tyler and 
Sundberg point out that "what young persons need is not just to make a 

^Max F. Baer, "Washington Flashes, " The Personnel and 
Guidance Journal , Vol. XLIII, No. 3 (November, 1964), pp. 228-229. 

^John W. M. Rothney, Guidance Practice and Results (New York; 
Harper, 1958), p. xx. 
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choice but to learn how to choose. Again, Tyler says, "a person to 

some extent shapes the pattern of his life by choices and decisions he 

makes at successive stages. Each satisfactory decision-making 

endeavor prepares the individual for making satisfactory decisions in 

the future. It is possible to help the student gain decision-making skills 

through counseling. The counselor can help the student identify 

alternative choices and weigh these choices to determine the choice 

with the greatest payoff. Gelatt writes, 

through ’’decision- making counseling” students are 
required to learn about themselves and their environ- 
ment as this information is related to the decision, 
and by participating in the decision-making process 
they can learn to make decisions more^independently 
and accept the proper responsibility. 

The area of decision-making in which the counselor and the 



student are most often involved is that of educational and vocational 

planning. Clarke and Gelatt write, "a major function of secondary 

guidance service is to aid in the educational and vocational development 

„11 

by facilitating good decisions concerning individual students. 



®Leona E. Tyler and Norman D. Sundberg, Factors Affecting 
Career Choices of Adolescents, Cooperative Research Project No. 
2455 (University of Oregon, 1964), p. 2. 

^Leona E. Tyler, ’’Theoretical Principles Underlying the 
Counseling Process, ” Journal of Counseling Psychology , Vol. 5 
(1958), p. 6. 

^^Harry B. Gelatt, op. cit . , p. 241. 

^ ^Robert Clarke and Harry B. Gelatt, Report on NDEA Guidance 
Research Project -- Palo Alto Unified School District, January, 1963, 
Technical Report, p. 4. 
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Yabroff^^ investigated the feasibility of teaching decision-making 
to ninth grade students in the area of vocational and educational choice. 
Two factors were important in preparing the student for decision-making: 
one was general information relating to colleges and occupations; and 
the other factor was specific information relating to how well students 
like him, from his school district, had done in post-high school jobs 
and college. The first order of alternatives was developed after 
receiving the general information. The choices were determined 
largely by the student's interest in various fields. The second order 
of alternatives was selected from the first by the process of not only 
knowing what the student would like to do but what the student i3 capable 
of doing in the light of what others like him have done. 

In Yabroff's study all subjects were given a four week general 
information course. One- third of the subjectL received an additional 
week of information -- comparing the students to national norms --, 
one- third of the subjects were given information relating them to local 
norms, and one-third no further training. The group with the informa- 
tion about local norms did significantly better in making realistic 
choices, as judged by the guidance staff, than the other two groups. 

The student in this group was able not only to form numerous 

^^William W. Yabroff, An Experiment of Teaching Decision- 
Making to Ninth Graders (unpublished paper. May, 1964). 



20 August 1965 



34 



TM -26 11/000/00 



alternatives (based on general information) but to determine which of 
the alternatives allowed him the best chance for success. 

Discussion 

The scene is rapidly changing for America's youth. Not only are 
there millions of young people more than in past generations, but 
educational and vocational values are changing as well. The influence 
of automation and the upward mobility of our society has made the 
professional and managerial job more desirable and the blue-collar 
and many of the white-collar jobs less desirable. This means more 
people will remain in high school and more will set their goals toward 
college. This also means that the non- academically oriented young 
person will have increasing difficulties in securing unskilled positions. 
Another complicating factor is the rise in juvenile delinquency which 
seems to be a concomitant of population growth. 

Not only is the number of students on the increase but the 
complexity of their problems as well. The scientific revolution has 
created a change in the nature, and increased the number, of jobs. 

In order to keep pace, education has had to increase the depth and 
breadth of its programs. The number of alternatives a student must 
analyze to make his decisions is greater today than ever before. The 
amount of information the student must have to select appropriate 
alternative^ is equal in magnitude to the scope of his alternatives. 
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The counselor must, in order to fulfill his responsibility to the 
students and society, be permitted to spend more time aiding students 
with their unique problems and less time with repetitive problems. It 
is not that repetitive tasks should be slighted, but that a more efficient 
way of dealing with them must be found. The counselor must also be 
provided with a way to utilize effectively greater quantities of available 
information and to dispense the increasing amount of pertinent infor- 
mation to the students. In order for the student to use the information 
effectively decision-making skills must be taught. 

The automated counseling system is a step in this direction. It 
can free the counselor of certain repetitive tasks, evaluate the 
information necessary to assess the student's chances for success, 
provide the students with information, and it can be useful as a tool 
in aiding the students to learn more effective decision-making. 

Computer Research 



Introduction 

Computers are commonly thought to be a mid- twentieth century 
development. However, the idea of a digital computer is rather old. 
A professor at Cambridge, Charles Babbage, planned a computer 
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1 3 

called the Analytic Engine in the early 1800's. Had the engine been 
completed it would have been a mechanical device, using wheels and 
cards, one hundred times, slower than the slowest electronic computer 
but decidedly faster in arithmetic calculations than man. 

The world of computers had its real growth in the 1940' s and 
1950' s. The use of computers has been, and still is, largely in the 
area of mathematical calculations. However, more research is being 
done in the areas of complex problem-solving and simulation of human 
behavior. 



Definitions of Terms and Concepts 



Algorithms and Heuristics . The terms algorithmic and 

heuristic methods of problem-solving are often encountered. These 

terms may be defined as follows: 

One very special and valuable property that a generator of 
solutions sometimes has is a guarantee that if the problem 
has a solution, the generator will, sooner or later, produce it. 
We will call a process that has this property an algorithm for 
that problem. The guarantee provided by an algorithm is not 
an unmixed blessing, of course, since nothing has been 
specified about the cost of time required to produce the 
solutions. For example, a simple algorithm for opening a 
combination safe is to try all combinations, testing each one 
to see if it opens the safe. This algorithm is a typical problem 
solving process; there is a generator that produces new combi- 
nations in some order, and there is a verifier that determines 



1 3 

Edward A. Feigenbaum and Julian Feldman, Computers 
and Thought (New York: McGraw-Hill, 1963), p. 16. 
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whether each new combination is in fact a solution to the 
problem. This search process is an algorithm because it is 
known that some combination will open the safe, and because 
the generator will exhaust all combinations in a finite interval 
of time. 

A process that may solve a given problem, but offers no 
guarantees of doing so, is called a heuristic for that problem. 

This lack of guarantee is not an unmixed evil. The cost inflicted 
by the lack of guarantee depends on what the process costs and 
what algorithms are available as alternatives. For most run- 
of-the-mill problems we have only heuristics, but occasionally 
we have both algorithms and heuristics as alternatives for 
solving the same problem . . . Sometimes, as in chess, every- 
one plays by heuristic, since no one is able to carry out the 
algorithm of examining all continuations of the game to 
termination. 

When the number of alternatives is large, humans use heuristic 

15 

methods. In playing checkers, for example, Samuel estimates that 

in order to explore every possible path it would take something like 
40 

10 moves. If it were possible to make three moves a millimicro- 

21 . 

second, it would take 10 centuries to exhaust all the possible 
combinations. Because of past checker- playing experience and knowing 
the rules of the game, the human player decides among several possible 
moves instead of attempting to analyze every possible move during the 
course of a game. 

In such decision-making activities as choosing a vocation or 
buying an automobile, heuristics are used since there its no 



^^Ibid. . pp. 114-115. 
^®Ibid., p. 72. 
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solution at the time of selection which is guaranteed to be the 
best. 

Simulation. In the process of simulating a human, heuristics 
are used. The cognitive process by which a counselor gives aid to the 
student is a heuristic one. since there are no known best solutions for 
the student's goals. The computer program written by analyzing the 
rules used by the counselor is also heuristic, since, again, no 
solution is guaranteed best. However, there are solutions that are 
better than others in the light of the student's capacities and perfor- 
mances. 

A distinction should be made between simulation and artificial 
intelligence. In artificial intelligence the goal is not to copy a human 
model in either the outcomes or in the process of arriving at the out- 
comes. The hope is that the program will not only do as well as the 
human attempting the same task but that it will do better. In 
simulation research the goal of the researcher is to duplicate that 
which is done by the human model. The researcher would be dis- 
appointed if the outcomes were better or worse -- they must be the 

same. 

The simulation project which this study has attempted to validate 
is not one of simulating the process of educational planning but of 
simulating the outcomes that were produced by the counselor. 



er|c 
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Research Related to the Development of an 
Automated Counseling System 



An application of the systems approach to the counseling 

function was cited by Cogswell. He defines systems analysis as 

. . . the analysis or study of a system. The procedures may 
involve the detailed descriptive analysis of an existing system, 
in which the various functions, the interactions between sub- 
systems, ;he flow of information, etc. , are analyzed in terms 
of their effect on total system performance. If the system to 
be analyzed doesn't actually exist, but has been proposed, a 
model of the tentatively planned system may be constructed by 
the use of simulation techniques. By simulating the system 
and its environments in systems research, functions and inter- 
actions can be varied and studies under controlled conditions. 

In the development of the automated counseling system a 
systems approach was used to analyze the pre- interview and interview 
behavior. The total system was subdivided into components which were 
analyzed as to their functions and to the interaction with other com- 
ponents. 

Cogswell found that a systems approach to a pupil personnel 

program was helpful in uncovering problem areas. 

Study of the data indicated analysis conducted from a systems 
frame of reference does uncover problem areas in a relatively 
short period of time, and that analysis of the total system 
would yield information that could be used to plan future 
development of the school district along more effective lines. 



John Cogswell, The Systems Approach as a Heuristic Method 
in Educational Development -- An Application to the Counseling 
Function, Systems Development Corporation (Santa Monica, 1962), 



pp. 2-3. 
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In addition, thinking of the role of counseling as a function 
within a large man-machine system was heuristic in producing 
hypothesej^for systems research in relation to the counseling 

function. 

Cogswell and Estavan have established automated interviewing 

techniques which are directly applicable, with the exception of the 

differences of the input -output device, for use in the research project 

18 

this study is attempting to validate. The laboratory (CLASS) is a 
computer-based system for individual instruction. A student is given 
a diagnostic question on a film viewer and selects his answers to each 
question on the student response console which has a multiple -choice 
mode. The computer in turn analyzes each response and records those 
indicated by the computer program. The computer then chooses the next 
question for the student on the basis of prior responses. The program 
with some modification is suitable for the interview aspect of the 



automated counseling system. 

The decision-making model of physicians has been adapted to 

19 

computers in the area of medical diagnosis (Smith ) . Ledley and 



^^John Cogswell and Donald Bushnell, A Computer-Based Laboratory 
for Automation in School Systems, SP-256, Systems Development Corporation 
(Santa Monica, March, 1961) . 

^^S. C. Vandenberg, H. F. Silberman, L. Uhr, C. F. Wrigley, 

W. H. Holtzman, and P. A. Smith, "Computers in Behavioral Sciences: 

The Impact of Computers on Psychological Research," Behavioral Science, 
Vol. 5 (1960), pp. 170-187. 
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Lusted have also studied the problem of using computers in medical 
diagnosis. Both of these diagnostic programs analyze patient histories 
and current physical status. The results are probability statements 
concerning the various possible diagnoses and are, in most cases, as 
accurate as those made by physicians. 

2 1 

An application in the field of psychology was made by Holtzman. 

In attempting to use the computer in several tasks in a clinical setting, 
he concluded that a computer cannot interact with the patient as well 
as the clinician, but the machine would help in: (1) collecting routine 
information, such as biographical details or Minnesota Multiphasic 
Personality Inventory responses; (2) processing test protocols; and 
(3) making certain types of interpretations based on actuarial pre- 
dictions. 

Although Holtzman, in the above study, found interaction between 
the patient and the machine not as good as between the patient and a 
clinician, this problem was not a liability in this study. The automated 
counseling system was not attempting to deal with problems as unique 
as those fou in patients seeing a clinician. However, if the counselee 

20 

Robert S. Ledley and Lee B. Lusted, "Computers in Medical 
Data Processing, " Operational Research, Vol. 8 (1960), pp. 299-310. 

21 



Vandenberg, et al. , op. cit. 
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had unique problems outside the scope of the automated interview, he 
had the opportunity to request an individual counseling session with the 
counselor. 

22 

Dr. Joseph Jaffe , a Harvard Research Fellow, has developed 
a computer program for the analysis of psychoanalytic interviews. He 
says the trained psychiatrist quickly learns to interpret what is in the 
interview, but often needs as many as ten interviews to determine and 
interpret what is not, the omissions, in the interview. The analysis 
of interviews, called psycho- linguistics, looks not only at overt content 
but at form, cadences, patterns, and omissions. What a computer in 
this process does is no different from what analysts may spend hours 
for each patient doing. Jaffe feels the addition of computerized 
principles to psychiatry would eliminate much of the time-consuming 
work being done. This would shorten the period of analysis, and would 
bring the help offered by the psychiatrists into the reaches of many 
more people. 

The notion of providing an extension of the psychiatrist by the 
addition of computerized techniques can be carried one step further 
to support the idea that computer techniques will also extend the 
counselor and allow him to serve more people adequately. 

22 

Flora Schreiber and Melvin Herman, "The Computer Age 
in Psychoanalysis, " Science Digest (January, 1965), pp. 18-19. 



20 Aug^st 1965 



43 



TM-2611/000/C0 



A program was written by Clarkson to simulate the selection 
process an investment broker goes through in selecting stocks for his 
client's portfolio. Certainly the work of a stock broker is not directly 
related to psychology or education. However, the steps and the logic 
employed in the selection process by the computerized investor 
parallels closely the program of the automated counseling system. 

It may be useful to look at a flow diagram of Clarkson's invest- 
ment model for the purpose of comparison with the automated 
counseling system (see the following page). The flow diagram for the 
automated counseling system may be found on page 41 of Chapter Three. 

Section (A) in the trust investment model is analogous to the 
student appraisal section of the automated counseling system. The 
investment model determines the nature of the economy, whereas 
the counseling model determines the nature of the academic world 
and the student's performance in it. The investment process in 
section (B) analyzes goals and policies of the client, while the auto- 
mated counseling system analyzes the student's plans and aspirations 
during the automated interview. The (C) process in the investment 
model, the actual selection of stocks, has no completely analogous 
computerized component in the automated counseling system. For 

23 

E. A. Feigenbaum and J. Feldman, op. cit. , pp. 347-371. 
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(B) 




Figure 1 
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the course selection is done by the student, with the computer inter- 
vening only when an inappropriate selection has been made. 

The results of test portfolios have been proxnising. For four 
clients the model investor chose 2540 shares of stock. The automated 
investing system chose a similar number with 2145 of the shares 
exactly the same as the model investor. This is an agreement of 84% 
with no attempt to determine if the machine made better or worse 
selections than the human model. 

It may be concluded, on the basis of these studies, that 
simula^^on of a process, such as educational planning, is possible 
with a high degree of accuracy. 
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CHAPTER III 
PROCEDURES 
Overview 

Intr oductior 

The purpose of this chapter is to describe the procedures used 
in determining the validity of the automated counseling system. How- 
ever, for purposes of perspective it is important to»: (1) summarize 
the objective of the larger research program of which this study ie a 
part; (2) outline the development and current specifications of the 
automated counseling system. These two tasks will be completed 
prior to discussing the procedures which are employed in this study. 

Summary of Problems and Objectives 

The problem of this study is the validity of a computer- b‘'sed 
counseling system, and the general objective is to determine the 
validity of this automated system. The specific objectives involved: 
(1) determining the agreement between the automated system and the 
model counselor in regard to pre- interview pupil appraisal, post 
interview pupil appraisal, educational decisions and completeness of 
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educational plans; (2) determining the agreement between the automated 
system and the second counselor in regard to pre- interview pupil 
appraisal, post interview pupil appraisal, educational decision, and 
completeness of educational plans; (3) comparing the differences of 
those students counseled by the machine and the second counselor 
regarding pre- interview pupil appraisal, post interview pupil appraisal, 
educational decisions, and comp!” teness of educational plans; and (4) 
identifying the specific limitations of the computer-based counseling 
system in regard to machine limitations, attention span, lack of re- 
inforcers, lack of sensitivity, lack of student cooperation with the 
machine, and reservations about course plans. In terms of the specific • 
objectives the following three major hypotheses are tested: (1) the auto- 
mated counseling system will not be significantly different from the 
model counselor regsi.rding pre- interview pupil appraisal, post- interview 
pupil appraisal, educational decisions, and completeness of educational 
plans; (2) the automated counseling system will not be significantly 
different from the second counselor regarding pre- interview pupil 
appraisal, post-interview pupil appraisal, educational decisions, and 
completeness of educational plans, and (3) the extent of agreement be- 
tween the automated system and the model counselor will not be significantly 
diiferent from the extent of agreement between the automated system and 
the second counselor regarding pre- interview pupil appraisal, post- inter- 
view pupil appraisal, educational decisions, and completeness of 



educational plans. 
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The Objectives of the Larger Research 
Program 

The concept of an automated counseling system was developed 
initially by Cogswell and Loughary. ^ The project of developing an 
automated counseling system to be implemented in educational planning 
of secondary students is part of a larger research program. The 
larger research program is investigating the use of man-machine 
systems in automated school systems. Automated school systems 
would use computer-based programs for individual and group instruc- 
tion, counseling, storage of school data, aid in decision-making 
situations, and administrative and clerical functions. 

Specifications of the Automated Counseling System 
Summary of Development 

In order to better understand this particular study concerning 
the validation of the automated counseling system, some background 
information about the development of this system may be helpful. 

After the initial preliminary study (described in the Background and 
Rationale section of Chapter One) indicated the development of an 
automated counseling system was feasible, the plan of operation was 

^University of Oregon Conference held Spring, 1963. 
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developed. The plan had four phases: (1) identifying the rules used 
by the counselor in assessing the student during the pre-interview 
examination of the student's cumulative folder, and reducing these 
rules to a format that could be programed for a computer; (2) identi- 
fying the rules used by the counselor during the actual interview, and 
constructing the machine interview format in a manner compatible 
with computerization; (3) translating the pre-interview and interview 
format into machine language; and (4) the process of analysis and 
comparison of the automated system to human counselors in inter- 
viewing situations to determine the validity of the automated instrument. 

Specifications of the Automated Counseling 
System 

Introduction. The counselor chosen as the model for the system 
(a detailed description of the counselor selection procedure is found on 
pages 6-7, Chapter One) was asked to select twenty- five students, all 
in the ninth grade, with whom he had not counseled prior to this study, 
for an educational planning interview. The students selected were to 
represent, in the counselor's estimation, the range of students typical 
of this junior high school. He recorded his pre-interview behavior 
for each of these students, i. e. , verbalized his thought as he studied 
each cumulative folder, and tape-recorded a typical educational 
planning interview for each student. 
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Typescripts were made of these recordings and then examined 
by the research team (see footnote number 6, Chapter One for 
members of the team). Flow charts describing the procedures involved 
were prepared for each typescript. Flow charts and typescripts them- 
selves were examined intensively by the researchers for the purpose 
of identifying rules which the counselor employed. 

The diagram on the following page dc ribes the system which 
has been designed. It can be seen that there are two primary out- 
comes towards which the program is working. These are labeled 
"A" and "B" in the diagram. Outcome "A" is a computer program 
which analyzes the cumulative folder data for students in terms of 
educational planning objectives. Outcome "B" is a computer program 
which in essence is an automated interview for the student, again in 
regard to educational planning. Before continuing with the description 
of the proposed project it may be helpful to comment briefly on the 
system as displayed on the flow chart. 

The diagram indicates that the first step in the system is to 
collect pupil appraisal data. These data are traditionally found in 
the student's cumulative folder. The next step is to code and store 
the data in a computer. Next, the computer analyzes the data 
according to rules used by the computer. Following this analysis 
the computer determines whether or not the student is ready for the 
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automated interview in regard to educational planning. In the event 
the student was not ready (for example, he may not have the current 
course grades in his folder, which would make it impossible for the 
computer to make educational success predictions), appropriate 
notification to the counselor is made. If it were determined the 
student was ready for the automated interview, the results of the auto- 
mated analysis of the cumulative folder are fed into storage in 
preparation for the automated interview. It is possible that a report 
to the counselor could be generated at this time. The focus w*!!! next 
be on the interview. 

Orientation . Prior to the interview each student was told the 
nature of the study and was given instructions on how to operate the 
input- output device. 

The students were told that they were helping in a research study 
to determine the effectiveness of a computerized interviewing system. 
They were also told that they would have two interviews, one with a 
counselor and one with the machine, and that both interviews would deal 
with their educational planning. Since the time lapse between the two 
interviews was only, at the minimum, 24 hours there may have been 
little wear- off effect. The students were made aware of this factor, 
and were asked to cooperate by using as little of the information gained 
in the machine interview as possible when talking with the counselor. 
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The input- output device is similar in operation to a typewriter. 
The students were informed how to enter their responses and how to 
signal for help with any operational difficulties that may a^ise during 
the miachine interview. 

The Interview. The automated counseling interview consisted of 
the student interacting with the computer according to a branching- type 
program designed to perform the functions performed by the model 
counselor. 

The student interacted with the computer by means of equipment 
already available to Systems Development Corporation. The equip- 
ment that was used was an IBM Q-32, a computer located at Systems 
Development Corporation in Santa Monica, and the TWX-- model 33, 
an input- output device located in Palo Alto. A telephonic circuit 
permitted communication between the two pieces of hardward in Palo 
Alto and Santa Monica. 

The goal of this simulated interview was to have a student 
select his tenth grade courses and make tentative selections for the 
eleventh and twelfth grades. During the interview the input- output 
device automatically wrote out instructions to the student. If the 
instructions were of a multiple- choice nature the studek.t responded 
by depressing one key, A to E, for example, in the case of a five- 
choice response. At other times the student was requested to type a 
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short sentence when individual expression is necessary. Typing 
accuracy on the part of the student was important only when one exact 
key was depressed as in the multiple- choice items. The format of 
the machine interview, which was taken from the rules extracted from 
the counselor's interviews, is reproduced in Appendix A. 

Machine Functions. The student selected courses based on his 
goals and preferences. The appropriateness of these selections were 
based in part upon the type of college the student planned to attend -- 
which he indicated during the machine interview -- whether or not he 
planned to attend college, his present GPA, and the minimum require- 
ments for graduation from high school. If the student planned to attend 
a specific type of college but didn't take enough special units the machine 
informed him of this deficiency. If the student planned to go to college 
and had very low grades, or didn't plan to go and had high grades the 
machine told him he was setting his sights a little high or a little low. 

The GPA was used to determine the number of solids (homework 
courses) he should enroll for, and whether he would be permitted to. 
take certain advanced courses. The student was not informed of these 
rules unless he made an inappropriate choice. The program stated the 
courses required for each grade, however, before the student selected 
the remaining courses. 
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Termination . During the automated counseling interview but 
before being asked if the selected courses are the final choices, the 
student had the opportunity to request an appointment with the counselor 
for further counseling. If he did not make the request, he was aslced 
if he was satisfied with the high school course program he had selected. 
If he was satisfied, these choices were then stored in the computer in 
preparation for student schedule building. If he was not satisfied, he 
was either allowed to change his course program or he would receive 
an automatic referral to the counselor. 

Procedures 

General Design 

The validity of the automated counseling system was determined 
by comparing the results of the automated system with those obtained 
through a regular interview with a counselor in regard to educational 
planning. The criterion that was used is whether or not the differences 
between the automated system and the counselor, in terms of the 
specific objectives, are significant. The statistical design employed 
in comparing the counselors with the automated system is a Type I 
design using the latin square; and the comparison between the model 
counselor- machine and the second counselor-machine groups is by a 
simple randomized design. 
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Sample 

A sample of forty ninth grade students was drawn at random 
from Wilbur Junior High School in Palo Alto, California. The students 
attending Wilbur are a high ability, middle socio-economic population. 
A student placing in the 50th percentile on the School and College 
Aptitude Test at Wilbur Junior High School would be in the 70th per- 
centile on national norms. This particular population is appropriate 
in that the automated counseling system was developed from observa- . 
tions of a counselor in that school district working with data from that 
district. The automated system was developed from a specific 
counselor working in a specific school environment with a particular 
system of appraisal data. To test the automated system using another 
school environment would introduce factors for which the computerized 
system has not been programed. 

Treatment of the Population 

Forty students were randomly divided into two groups of twenty 
students. Both groups had an interview with the automated counseling 
system. Group I also had an interview with the model counselor, and 
Group II had an interview with the second counselor. In order to 
combat the effect of temporal positioning of the interviews, such as 
always having the machine interview first, half of Group I and Group 
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II was counseled first by the machine, and the other half first by the 
counselor. 

The first task in data analysis is to determine how well the 
automated system simulates the outcomes of the model counselor. In 
Group I (see the diagram on the following page) the results of the 
automated interviews from both subgroups are compated with results 
from the counselor's interview. 

The second task is to determine how well the automated counsel- 
ing system simulates the outcomes of the second counselor. In Group 
II the results of the automated interview from both subgroups are 
compared with the results from the counselor's interviews. 

The third task is to determine if the amount of agreement 
between the automated system and the model counselor was significantly 
different from the amount of agreement between the automated system 
and the second counselor. This is done by comparing the amount of 
agreement between the two counseling modes in Group I with those of 
Group II. 

The fourth task is to determine the effects of temporal positioning 
of the counselors in relation to the machine interview. Before the 
hypotheses related to the first three tasks could be accepted it was 
necessary to determine if the order in which the counseling modes 
were presented effected the outcomes. If the differences between 
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TREATMENT SCHEDULE 
GROUP I 



Subgroup lA 
N = 10 



Subgroup IB 
N = 10 



Treatment I 



Treatment II 




GROUP II 



Subgroup IIA 
N = 10 



Subgroup IIB 
N = 10 



Treatment I 



Treatment II 




Figure 3 
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II was counseled first by the machine, and the other half first by the 
counselor. 

The first task in data analysis is to determine how well the 
automated system simulates the outcomes of the model counselor. In 
Group I (see the diagram on the following page) the results of the 
automated interviews from both subgroups are compated with results 
from the counselor's interview. 

The second task is to determine how well the automated counsel- 
ing system simulates the outcomes of the second counselor. In Group 
II the results of the automated interview from both subgroups are 
compared with the results from the counselor's interviews. 

The third task is to determine if the amount of agreement 
between the automated system and the model counselor was significantly 
different from the amount of agreement between the automated system 
and the second counselor. This is done by comparing the amount of 
agreement between the two counseling modes in Group I with those of 
Group II. 

The fourth task is to determine the effects of temporal positioning 
of the counselors in relation to the machine interview. Before the 
hypotheses related to the first three tasks could be accepted it was 
necessary to determine if the order in which the counseling modes 
were presented effected the outcomes. If the differences between 
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seeing the counselor first and seeing the machine first were significant 
even though the differences between treatments were not, it could not 
be assumed the machine was simulating the counselor. The effects 
of position for those students seeing the model counselor is deterrhined 
by the agreement between the automated system and the counselor 
within subgroup lA, compared with the agreement within IB. The 
effects for the student seeing the second counselor are analyzed in the 
same manner using subgroups IIA and IIB. The total effect of position- 
ing are determined by combining subgroups lA and IIA and comparing 
them with subgroups IB and IIB. 

These four tasks are studied in terms of the specific objectives 
1 and 2 on page 13 in Chapter One. 

A fifth task is to determine some possible limitations of an 
automated counseling system (see specific objective 4, page 14 
Chapter One). All forty students are considered as one group when 
studying this task. 



Data Collection 



Data Collected 

The Automated Counseling System and the Model Counselor . 
1. Pre- interview Pupil Appraisal. Two types of appraisal 
statements were collected from, an analysis of the student's 
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cumulative records: (1) a statement concerning the student's pre- 
dicted high school GPA based on a follow-up study involving students 
from the Palo Alto Unified School District; and (2) statements relating 
to the student's academic status, potentials, and problems based on 
counselor rules developed in the preliminary study. 

The first type of pupil appraisal was stated in terms of contin- 
gency statements. After analyzing the student's cumulative folder, 
which was done just prior to the automated counseling interview, the 
computer printed out statements regarding the probability of a student 
earning a GPA of 2. 00 or less, 2.00-2.49, 2.50-2.99, 3.00-3.49, 
and 3.50-4.00 in high school. The predictions about the student were 
on the basis of eleven decile groups -- zero out of ten chances to ten 
out of ten chances. 

Prior to the counseling interview the counselor made a similar 
group of statements concerning GPA (see page 170 , Appendix B, for 
the forms to be used to collect these data). The second type of 
statement, based on rules taken from the model counselor's pre- inter- 
view behavior during the rule- building phase of the study, was respond- 
ed to by the machine and the counselor. A check list of fourteen 
statements relating to the student's present and future adademic status 
(see page 171, Appendix C) was collected from the counselor after he 
had indicated which statements applied to the student. The computer. 
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after examining the cumulative record, printed out which statements 
from the list of fourteen applied to the student. 

The data that were collected were the predicted GPA and the 
appraisal statements which apply to the student from both the machine 
and the counselor. 

2. Post- interview Pupil Appraisal. During the interview the 
counselor evaluated the realism of the student's appraisal of his own 
ability, his plans after high school, and his course selections (see 
Appendix D). The computer also printed out a similar group of state- 
ments. Both the statements from the counselor and the computer were 
collected. 

3. Educational Decisions. After each interview, one with the 
machine and one with the counselor, the student selected his courses. 
The student's course schedule after each interview was collected. The 
course schedule generating from the machine interview was a machine 
print- out listing by title the courses planned for high school. The 
schedule from the counselor's interview was a form on which the 
student has placed his courses by title for high school (see Appendix 

D for student's course schedule blank). 

4. Completeness of Educational Plans. The outcomes of both 
the automated and counselor interviews were the course plans of the 
student. Most of the students made total plans for thveir high school 
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courses. Others made partial plans but could not decide what courses 
to take for the remainder. And there were some students who were 
unable to select courses at all. 

The number of students who had complete course plans, the 
number who had partial course plans, and the number who had no plans 
at all, were collected for each of the two interviewing modes. 

The Automated Counseling System and the Second Counselor . To 
be practically useful, the automated counseling system must not only 
simulate the outcomes of a mode] counselor, but must also closely 
approximate what other expert counselors in the same school district 
are doing. For the purposes of determining the extent to which a 
computerized system, based on one model counselor, could be used in 
a school district with many counselors, a second counselor from the 
model counselor's district was included in the study. 

The data collected for Group II were the same as that collected 
for Group I in regard to the following: 

1. Pre- interview Pupil Appraisal, 

2. Post- interview Pupil Appraisal, 

3. Educational Decisions, and 

4. Completeness of Educational Plans. 

The Comparison of Group I with Group II. After all data fcr 



Group I and Group II had been collected and analyses made for each 
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group, the two groups were compared by determining the degree to 
which the automated counseling system produced results comparable 
to each of the two counselors. 

1. Pre- interview Pupil Appraisal. The data that was collected 
was the same as that collected for the independent analysis of Group 1 
and Group 11 except that subgroup lA was combined with IB and 11 A with 
IIB. 

2. Post- interview Pupil Appraisal. The data that was collected 
was the same as that collected for the independent analysis of Group 1 
and Group 11 except that subgroup lA was combined with IB and llA with 
IIB. 

3. Educational Decisions. The combined scores of lA and IB 
was collected and was compared with the combined scores of llA and 
UB in terms of educational decisions. 

4. Completeness of Educational Plans. The combined scores 
of subgroups lA and IB and the scores of llA and IIB relating to the 
degree to which students had made total course plans, partial course 
plane, and no course plans was collected. 

Specific Limitations of the Computer-based Counseling System. 
There are some qualities which may exist in the relationship between 
two individuals, such as in a regular counseling interview, that may 
be partially or wholly lacking in the machine interview. Since the 
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student v/as having the machine interview, some of the possible weak- 
nesses were apparent to him. The data collected was from analysis 
of interview performance and comments from the students at the close 
of the interview. 

1. Machine Limitations. The students who were unable to make 
total course plans were asked these questions; (a) Was your inability 
to make total course plans a result of the machine not presenting the 
material clearly? and (b) Did you feel the machine did not give you 
enough information to make adequate choices ? 

2. Attention Span. The number of students who terminated the 
interview early were collected. Also, each student was asked if the 
interview bored him or caused him to be restless. 

3. Lack of Reinforcers. The students were asked if they felt 
the machine not informing them about whether their response to each 
item was good or poor made it more difficult to make selections. They 
were also asked if the lack of' the presence of a counselor adversely 
affected their decision- makings. 

4. Lack of Sensitivity to the Student's Needs and Orientation 
towards Counseling. The students were asked if the machine interview 
met their needs in terms of high school course planning, and if it met 
their expectations of what an educational planning interview should be. 
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5, Lack of Student Cooperation. The number of students who 
attempted to jam the machine and the number of students who wrote 
in ridiculous responses were collected. 

6. Reservations About Course Plans. The number of students 
who had reservations about their courses chosen during the machine 
interview and during the counseling interview were collected. 

How the Data were Collected 

The Automated Counseling System and the Model Counselor . 

1, Pre- interview Pupil Appraisal. At the end of the pre- 
interview analysis, the computer printed out predictions concerning 
the student's high school GPA and appraisal statements relating to 
present and future academic status of the student. This print- out was 
collected. 

The counselor will make a similar analysis by completing the 
forms shown in Appendix B and C. These forms were collected after 
the counselor had analyzed the student's cumulative record. 

2. Post- interview Pupil Appraisal. The computer printed out 
data relating to the realism of the student's plans and choices. This 
print- out was collected. 

The counselor completed a form (Appendix D) giving similar 
data, which was collected after the interview. 
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3. Educational Decisions. After the machine interview the 
machine printed out the student's course schedule. After the interview 
with the counselor the student completed, in writing, a course schedule. 
These two schedules were collected. 

4. Completeness of Educational Plans. An analysis of the 
student's course schedule showed if he had made total, partial, or no 
course plans. The course schedule planned after the interview with 
the counselor indicated total, partial, or no planning. 

The Automated System and the Second Counselor . The data v»ere 
collected in the same manner for Group II as was used for Group I in 
terms of: 

1. Pre- interview Pupil Appraisal, 

2. Post- interview Pupil Appraisal, 

3. Educational Decisions, and 

4. Completeness of Educational Plans. 

The Comparison of Group I with Group II. 

1. Pre- interview Pupil Appraisal. The method used in collecting 
the data was the same as the method used for the machine and the 
model counselor and the machine and the second counselor analyses. 

2. Post- interview Pupil Appraisal. The method used in col- 
lecting the data was the same as the method used for the machine and 
the model counselor and the machine and the second counselor analyses. 
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3. Educational Decisions. The same procedure was followed in 
collecting the data for educational decisions was used for pupil appraisal 

data. 

4. Completeness of Educational Plans. The same collection 
procedures was used as for the collection of pupil appraisal data. 

Specific Limitations of the Computer-based Cou nseling System. 

1. Machine Limitations. These questions were asked in a short 
structured interview at the close of the machine interview. The 
questions asked were included in a more lengthy structured interview, 
which was given by another researcher investigating another problem 
in regard to the automated counseling system. Rather than have two 
structured interviews, the few questions necessary for this study were 
included in the interview of the other researcher (see Appendix F for 

the questions pertaining to machine limitations). 

2. Attention Span. An analysis of the machine interview indicated 

if the student terminated early, since the print- out of the machine 
interview would not be complete. The question regarding boredom 
was asked in a structured interview at the end of the machine inter- 
view. 

3. Lack of Reinforcers. These questions were asked in a 



structured interview. 
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4. Lack of Sensitivity to the Student’s Needs and Orientation 
towards Counseling. These questions were asked in a structured 
interview. 

5. Lack of Student Cooperation. An analysis of the interview 
print- out showed those who tried to jam the machine or who typed ‘n 
inappropriate responses. The number of students involved in either 

of these activities was recorded. 

6. Reservations About Course Plans. The form on which the 
students entered their high school course plans during the counselors' 
interview contained a question which asked if they had any reservations 
about these course plans. During the structured interview which 
followed the interviews of both counseling modes the students were 
asked if they had any reservations about their courses chosen during 
the machine interview. The number of students who indicated 
reservations about course plans selected during either interviewing 
mode was recorded. 



Data Analysis 

The analysis of the data is done by using a Type I design. This 

consists of using a latin square (see below) 

A1 A2 

Bi I I 

B2 " 
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in which A = Treatment, A1 = Machine and A2 = Coimselor; and 
B = Order, B1 = Machine interview first and B2 = Counselor's inter- 
view first. A Type 1 design is important in that it counterbalances 
the effects of the temporal positioning of the treatment orders (B1 and 
B2). This design makes it possible not only to study the effects of 
ordering upon the treatments but also the interaction of ordering and 
treatments. 

The analysis of the total sum of squares is as follows: 

SSt— SSs 4* SSv^s 

SSt=SSb + SSwb + SSws 

SSt=:SSa + SSs + SSas 

SSt=SSa + SSb + SSab + SSw cells 

The extent to which the machine (Al) simulated the counselor 
(A2) is determined by whether or not the F=MSa/MSerror (w) is 
significant. 

Similarly, the effects of ordering upon the two treatment modes 
are tested by determing if F=MSb/MSerror (b) is significant. 

The interaction effect between the treatments and order, A and 
B, is determined by F=MSab/MSerror (w). If the difference is 
significant the particular combination of treatment and order had 



effected the outcomes. 
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The F ratio is considered significant at the .05 level for all 
analyses. 

The Automated Counseling System and 
the Model Counselor 

Pre- interview Pupil Appraisal. The probability statements 
relating to GPA were made in the form of how many chances out of 
ten the student has for reaching a specific GPA in high school. For 
the criterion measure, GPA, there were eleven categories -- zero 
out of ten to ten out of ten chances. The category in which the student 
has the greatest number of chances was selected to represent the 
appraisal of the student, and its number, one to eleven, was placed 
in the latin square. If there were two equal larger numbers of chances 
the means of the two categories were used. 

Students in subgroup lA had their scores placed in the AiBl 
square (of the latin square) for the machine interview and in A2B1 
for the counseling interview. Student scores in subgroup IB were 
placed in square A1B2 for the machine interview and A2B2 for the 
counseling interview. 

This data are analyzed to determine if the F ratio between the 
machine and the counselor was significant. 

In the analysis of appraisal statements concerning the student's 
present and future academic status a value of one was assigned if the 
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statement applied to the student and a value of two if it did not. A 
separate analysis was done for each of the fourteen statements. 

Students in subgroup lA had their scores (the value assigned to 
the statement). placed in the AlBl square for the machine interview 
and in the A2B1 square for the counseling interview. Student scores 
in subgroup IB were placed in square A1B2 for the machine interview 
and A2B2 for the counselor's interview. 

Post- interview Pupil Appraisal . Statements relating to how 
realistically the student views himself (see Appendix D) were assigned 
a value of one if the answer of yes was given to the statement and a 
value of two if the answer was no. A separate analysis is done for 
each of the five statements. 

Students in subgroup lA had their scores (the value assigned to 
the statement) placed in square AlBl for the machine interview and 
in A2B1 for the counseling interview. Student scores in subgroup IB 
were placed in square A1B2 for the machine interview and A2B2 for 
the counselor's interview. 

Educational Decisions . The criterion measure is the differences 
between the courses selected by the student when counseled by the 
machine and when counseled by a counselor. The courses selected 
by the counselor are considered the basis for comparison. 
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Since the courses selected after being interviewed by the counselor 
were for the basis of comparison, each student in subgroup lA had the 
total number of courses selected after the counseling interview placed 
in square A2B1. The number of courses selected during the machine 
interview which agree with the course titles selected during the counsel- 
ing interview was placed in square AlBl. For example, if the student 
selected three courses after each of the two interviewing modes but 
only two of the course titles selected during the machine interview 
matched those selected during the counseling interview, the number 
three was placed in square A2B1 and the number two was placed in 
square AlBl. The same procedure applied for subgroup IB except 
for using squares A1B2 and A2B2. If the differences are not significant 
this indicates the actual courses selected for both interviewing modes 
are the same. A separate analysis is done for tenth, eleventh, and 

twelfth grades. 

When there are differences between the two interviews an attempt 
is made to determine if the differing courses fall into a specific pattern 
or category of courses. The courses are divided into three categories: 
(a) special unit courses -- special college entrance requirements; (b) 
non- special unit solids -- homework type courses; and (c) elective 
courses. The percentage of changing from one catagory to another, 
from a solid to a special unit, from an elective to a special unit, etc. , 
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was calculated. An analysis is also done to determine if course 
changes were made in the same subject area or if students changed 
from one area to another. 

Completeness of Educational Plans. The analysis involved the 
number of students making total course plans, the number making 
partial course plans, and the number making no course plans. The 
student was assigned a score of 1 if he had made total plans, a score 
of 2 for partial plans, and a score of 3 for no plans. The scores for 
students in lA for the machine interview are placed in square AlBl and 
for the counseling interview in A2B1. Subgroup IB is treated 
similarly using squares A1B2 and A2B2. The same procedure 
is repeated for the other three criterion measures. A separate 
analysis is done for the tenth, eleventh and twelfth grades. 

If the differences between the two interviewing modes are 
significant at the .05 level for any of the three criterion measures a 
further analysis is done. This analysis attempts to investigate 
whether individual student differences. School and College Aptitude 
Test scores, sex, and socio-economic variables are responsible. 
Comparisons are made between males and females, upper and 
lower groups in regard to scholastic ability using the mean of national 
SCAT norms as the dividing point, upper and lower groups using the 
mean of local SCAT norms, and three socio-economic levels based 
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on fathers occupation -- professional and managerial, semi-profes- 
sional, and laborer. These analyses involve the same procedures 
as stated for the criterion measures with the exception of subgrouping 
the population to correspond with the factors tested. Group I and 
Group U are combined and treated as one group for those items in 

which no significant differences occurred between the model counselor 
and the second counselor. 

The Automated System and the 
Second Counselor 



The data analyzed and the procedures used in the analysis of the 
data for group II, those interviewed by the machine and the second 
counselor, are the same as for those interviewed by the machine and 
the model counselor -- Group I -- in regard to the following; 

!• Pre- interview Pupil Appraisal, 

2. Post- interview Pupil Appraisal, 

Educational Decisions, and 
4. Completeness of Educational Plans. 

If the differences between the two interviewing modes are 
significant at the .05 level for any of the above four validity measures, 
a further analysis (see page 63, Chapter Three) is done. 
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The Comparison of Group I with Group II 

The design used for the comparison of Group 1 with Group II is a 
simple randomized design. 

The criterion measures that are used for the first analysis are 
the differences between the machine scores and the counselor scores 
used in the separate analysis of Group I and Group U, The deviation 
scores of Group I are compared to those of Group II. The second 
analysis combines the deviation scores of all those seeing the counselor 
first and comparing with those seeing the machine first. 

The first analysis is done by combining subgroups lA and IB 
and comparing with IIA and lib. This indicates the extent of agreement 
between the model counselor and the machine with the agreement 
between the second counselor and the machine disregarding the effects 
of order. Analyses are done for: 

1. Pre- interview Pupil Appraisal, 

2. Post- interview Pupil Appraisal, 

3. Educational Decisions, and 

4. Completeness of Educational Plans. 

The analysis of each item in these four validity measures yields a 
single F ratio. If the F is significant for that item then the difference 



o 



between the machine and the counselor is not attributable to chance. 
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A second analysis is done to determine if the students who saw the 
counselor first had results that varied significantly from the results of 
the students who saw the machine first. In this case students from both 
counselors who had the machine interview first are combined and have 
their scores compar -d with the scores of the students who saw the counselor 
first. A single F ratio for each item indicates if there are any signifi- 
cant effects on ordering for the sample population as a whole disregard- 
ing the effects of the two counselors. Analyses are done for: 

1. Pre- interview Pupil Appraisal, 

2. Post- interview Pupil Appraisal, 

3. Educational decisions, and 

4. Completeness of Course Plans, 

Mac hine Limitations 

1. Machine Limitations. The percentage of students who felt 
their inability to make total course plans was a result of the machine 
not presenting the material clearly, and the percentage feeling the 
machine did not give enough information for making adequate choices 
is calculated. 

2. Attention Span. The percentage of the students who 
terminated the interview early and the percentage who said they were 



bored is calculated. 
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3» Lack of Reinforcers. The percentage of students who felt 
not having the machine inform them as to whether an answer or 
response was good or poor made it difficult to seisct a course prograixif 
and the percentage who felt the lack of presence of the counselor caused 
them to be unable to complete a course plan is computed. 

4. Lack of Sensitivity to the Student's Needs and Orientation 
towards Counseling. The percentage of students who felt that the 
machine interview met their needs in terms of high school course 
planning and who felt the automated system met their expectations of 
what an educational planning interview should be is calculated. 

5. Lack of Student Cooperation. The percentage of students 
who tried to jam the machine, and the percentage who wrote in 

ridiculous responses is computed. 

6. Reservations about Course Plans. The percentage of students 
who had reservation about their course plans chosen during the 
counseling interview and the percentage who had reservations about 
their computer chosen course plans is computed. 
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CHAPTER IV 
RESULTS 
Introduction 

This chapter reports the results of the procedures used in 
determining the validity of the automated counseling system. 

The problem with which this study is concerned is the validity 
of an existing computer-based counseling system. The general 
objective is to determine the validity of this automated system. The 
specific objectives involve the following hypotheses. 

1. The automated counseling system will not be significantly 
different from the model counselor regarding; 

a. pre- interview pupil appraisal, 

b. post- interview pupil appraisal, 

c. educational decisions, and 

d. completeness of educational plans. 

2. The automated counseling system will not be significantly 
different from the second counselor regarding: 

a. pre- interview pupil appraisal, 

b, post- interview pupil appraisal. 
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c. educational decisions, and 

d. completeness of educational plans. 

3. The extent of agreement between the automated system and 
the model counselor will not be significantly different from the extent 
of agreement between the automated system and the second counselor 
regarding; 

a. pre- interview pupil appraisal, 

b. post- interview pupil appraisal, 

c. educational decisions, and 

d. completeness of educational plans. 

To offset the effects of having the students consistently inter- 
viewed by one counseling mode before the other, i. e. , the automated 
interview before the counselors' interview, half of the students were 
interviewed first by the machine and then by a counselor and half by 
a counselor first and then by the machine. Before any of the three 
hypotheses could be accepted it was necessary to determine if the 
lack of significant differences was related to the order of the inter- 
view modes or to interaction between the treatnaents and order. 

In addition to testing the three major hypotheses specific 
limitations of the computer-based counseling system were investigated 
in regard to (1) machine limitations, (2) attention span, (3) lack of 
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reinforcers. (4) lack of sensitivity. (5) lack of student cooperation 
with the machine, and (6) reservations about course plans. 

The results are reported in the order in which the three major 
hypotheses are listed. Before the limitations of the computer-based 
counseling system are reported, further analyses using the results 
obtained in the testing of the three hypotheses and other relevant data 
are discussed. This involves the following analyses. 

1. A comparison between the counselors and the machine using 
the number of students making total plans as the criterion measure. 

2. A comparison between the experimentally chosen tenth grade 
course plans (those chosen during the machine and the counselors' 
interview) and the tenth grade courses for which the students officially 
registered two weeks following the collection of data. 

3. A determination of the nature of course changes by comparing 
the courses selected during the counselors' interview with the courses 
selected during the machine interview for all three high school grades, 
the machine chosen tenth grade courses with final tenth grade courses. 

and the counselor chosen tenth grade courses with final tenth grade 
courses. 

4. An analysis of the items which indicated significant 
differences between counselors and the automated counseling system 




20 August 1965 



81 



TM-2611/000/00 



regarding sex, national and local School and College Ability Test 
norms, and family socio-economic level. 

The significance level used in this study, is .05. Any of the 
hypotheses were accepted if the differences were not significant beyond 
this level. 



Results 

The Automated Counseling System and 
the Model Counselor 

Hypothesis 1. The automated counseling system will not be 
significantly different from the model counselor regarding: 

a. pre- interview pupil appraisal, 

b. post- interview pupil appraisal, 

c. educational decisions, and 

d. completeness of educational plans. 

1-a. Pre- interview pupil appraisal. Prior to the actual course 
planning interview the model counselor and the machine predicted each 
student's high school GPA and made statements about students based 
on cumulative record folder data. Table 1 presents the analysis of 
these data. 

The hypothesis that there would be no significant differences 
between the model counselor and the machine was accepted for the 
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following items. For these items the machine and the counselor 

appraised each student similarly. 

Statement 1. Student's grades have gone down quite a bit. Ask 
about this in interview- -there may be personal problems. 

Statement 2. Student's grades have gone up. Ask about this in 
the next interview. 

Statement 3. This student should be watched closely. 

Statement 6. Low counseling priority. No problems apparent. 
Statement 8. Student is not achieving as well as aptitude scores 
would predict. 

Statement 9. Cumulative folder contains no aptitude scores for 

this student- -try to get some. 

Statement 10. Look out for over ambitious plans. 

Statement 11. Student may need to strengthen quantitative skills 
or will prob. bly experience academic difficulty. 

Statement 12. Student should improve verbal skills. If not, 
student may not be able to attain desired academic goals. 

Statement 14. Cumulative folder contains no grades for this 
student. 

The hypothesis that there would be no significant differences 
between the model counselor and the machine was rejected for five 
items. Regarding the following five items the machine and the counselor 
did not appraise each student similarly. 
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Predicted high school GPA. 

Statement 4. Student is a potential drop-out. 

Statement 5. Should be headed for college- -encourage student 

to explore widely in academic areas. 

Statement 7. Student is getting better grades than one would 

expect from looking at aptitude scores. 

Statement 13. The disparity between aptitude scores and 
achievement is so great that one is led to suspect aptitude test. Check 
test results. 

The direction of the differences for these items were; the 
counselor predicted lower GPA's, saw fewer students as potential 
drop-outs, saw fewer college bound students w ho should explore widely 
academic areas, found fewer students getting better grades than 
aptitude scores would predict, and saw fewer cases of great disparity 
between aptitude scores and achievement than did the machine. 

Since the pre- interview pupil appraisal was done prior to the 
interview there was no order or interaction effects to be tested. 

1-b. Post- interview pupil appraisal. After each interview 
questions pertaining to the content of the interview (problems, plans, 
and appropriateness of plans) were answered by the counselor using 
a check list. Similar data from the machine were obtained by 
inspection of the interview print- out. The results of the data analysis 
are given in Tables 2, 3, and 4. 
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The hypothesis of no significant differences between the model 
counselor and the machine was accepted for the following four questions. 

Question 1. Does the student indicate he is having problems with 
his courses ? 

Question 2. Does the student indicate he is going to college? 

Question 3. Is he planning to take more solids than his GPA 
warrants ? 

Question 4. On the basis of his projected grades and the type 
of college he is planning to attend (or not to attend college or trade 
school at all) is the student's planning realistic, aiming too high, or 
aiming too low ? 

The hypothesis was rejected for the following question. 

Question 5. What is the student's proposed college major: 
business, engineering or one of the sciences, social science, language, 
music and art, education, or student is not college bound? 

While on the basis of comparison between treatments (the model 
counselor versus the machine) there were no significant differences, 
the effect of the order of interview presentation was highly significant 
for Question 1. The students indicated significantly more problems 
with courses when the machine interview was first. Therefore, the 
hypothesis that there will be no significant differences between the 
model counselor and the machine regarding Question 1 must be rejected. 
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There were no significant order or interaction effects for any of 
the other items. 

1-c. Educational decisions. Educational decisions consist of 
specific course selections made by the student during his interview 
with the counselor and the machine. The courses chosen during the 
counselor's interview were used as the basis for comparison with 
the machine interview. As explained in detail in Chapter 3, the 
comparative data represents the number of specific courses selected 
during the machine interview which agree with specific courses selected 
during the counselor's interview. 

Only the students who completed at least partial course plans 
were compared. The vaxiance caused by those students not making 
any course plans is determined in the section reporting the differences 
between the counselor and the machine in terms of completeness of 
course plans. Only the courses which involved deci7^lbn- making were 
compared, i. e. , the required courses were not compared. 

As can be seen in Table 5 the hypothesis that there would be no 
significant differences between the model counselor and the machine 
regarding the specific courses selected for high school was accepted 
only for grade eleven. 

The hypothesis was rejected for grades ten and twelve. 



TABLE 5 

TABLE OF TREATMENT F'S FOR HIGH SCHOOL COURSE PLANNING COMPARING THE AUTOMATED 

COUNSELING SYSTEM WITH THE MODEL COUNSELOR 
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As shown in Table 6 there were no order effects. 

There were, however, significant interaction effects between 
order and treatments for grades ten and twelve, as can be seen from 
Table 7. For the tenth grade the mean number of counselor selected 
courses for subgroup lA (machine first) was higher (3.6) than the mean 
for subgroup IB (counselor first) while the mean for machine selected 
courses for subgroup lA was lower (2. 5) than the mean for subgroup 
IB (2. 7). In other words, the extent of agreement between the counselor 
and the machine was greater when the counselor was first than when 
the machine was first. Further analysis showed the differences 
between the courses selected during the counselor's interview and 
those selected during the machine interview to be significant for 
subgroup lA, but not for subgroup IB. In regard to interaction for 
grade twelve, the mean for counselor selected courses for subgroup 
lA was higher (3. 6) than the mean for subgroup IB (3.29) while the 
mean for the machine courses for subgroup LA was lower (2.2) than 
the m^ean for subgroup IB (2. 57), thus indicating closer agreement 
between the counseling modes regarding subgroup IB than subgroup lA. 
Further analysis showed the differences between the courses selected 
during the machine interview for subgroup lA to be significant beyond 
the . 05 level. For subgroup IB, the differences were also significant 

beyond the .05 level. 
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Because of the manner in which these data were treated it was 
possible for the student to have selected five courses during the 
machine interview and three courses during the counselor's interview 
and have three of the five machine courses match, in terms of course 
title, the three counselor chosen courses. In analyzing the data the 
machine would appear better than it really was since the data summary 
would indicate that three courses chosen by the machine matched three 
courses chosen in the counseling interview when, in fact, three out of 
five courses matched the three counselor chosen courses. Even 
though the frequency and the magnitude of this phenomenon was very 
low, a percentage of agreement between the counselor and the machine, 
disregarding order, was determined by taking into consideration the 
number of courses selected during the machine interview, the number 
selected during the counselor's interview and the number identical 
courses selected during both interviews. 

The percentage of agreement was 66 percent for the tenth grade 
courses, 65 percent for the eleventh grade courses and 53 percent for 
the twelfth grade courses. 

1-d. Completeness of course plans . While each student was 
asked to complete a course plan for the tenth, eleventh, and twelfth 
grades, only part of the sample were able to do so. Others could 
make only partial plans, and some could make no plans at all. The 
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students making total, partial, or no course plans during the machine 
interview were compared with the students making total, partial, or no 
course plans during the counselor's interview. Separate analyses were 
done for each grade level. The results are reported in Tables 8 , 9* 
and 10. 

The hypothesis that there would be no significant differences 
between the model counselor and the machine in terms of students 
making total, partial, or no course plans was accepted for the tenth 
and eleventh grades. 

The hypothesis was rejected for the twelfth grade. 

The machine did not successfully simulate the counselor regard- 
ing completeness of course plans for the twelfth grade. All the students 
seeing the counselor were able to make at least partial course plans 
while 40 percent of the students were unable to make any course plans 
at all during the machine interview. 

There were no significant order or interaction effects. 

The Automated Counseling System and 
the Second Counselor 

If the automated counseling system is to have practical use in a 
school setting it must not only simulate the counselor after whom it 
was modeled, but the automated system should also produce results 
similar to those of other counselors working with the same students 
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and using the same data. This section determines the agreement 
between the automated counseling system and a second counselor from 
the same school district as the model counselor. Data collection and 
analyses procedures employed are the same as for the automated 
system versus the model counselor. 

Hypothesis 2. The automated counseling system will not be 
significantly different from the second counselor regarding: 

a. pre- interview pupil appraisal, 

b. post- interview pupil appraisal, 

c. educational decisions, and 

d. completeness of educational plans. 

2- a. Pre- interview pupil appraisal . The hypothesis that there 
would be no significant differences between the automated counseling 
system and the second counselor was accepted for the following items 
as can be seen in Table 11. For these items the machine and the 
counselor appraised each student similarly. 

Predicted high school GPA. 

Statement 1. Student's grades have gone down quite a bit. Ask 
about this in interview- -there may be personal problems. 

Statement 2. Student's grades have gone up. Ask about this in 



the next interview. 
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Statement 3. This student should be watched closely. 

Statement 5. Should bcs headed for college- -encourage student 
to explore widely in academic areas. 

Statement 7. Student is getting better grades than one would 
expect from looking at aptitude scores. 

Statement 8. Student is not achieving as well as aptitude scores 
would predict. 

Statement 9. Cumulative folder contains no aptitude scores for 
this student- -try to get some. 

Statement 10. Look out for over ambitious plans. 

Statement 11. Student will need to strengthen quantitative skills 

.j , , n ti .t j / .» ■ 

or will probably experience academic difficulty. 

Statement 12. Student should improve verbal skills. If not, 
student may not be able to attain desired academic goals. 

Statement 14. Cumulative folder contains no grades for this 
student. 

The hypothesis that there would be no significant differences 
between the second counselor and the machine was rejected for three 
items. Regarding the following three items the machine and the 
counselor did not appraise each student similarly. 

Statement 4. Student is a potential drop-out. 

Statement 6. Low counseling priority. No problems apparent. 
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Statement 13. The disparity between aptitude scores and 
achievement is so great that one is led to suspect aptitude test. Check 
test results. 

In regard to direction of differences the second counselor saw 
fewer students as potential drop-outs, more students with a low 
counseling priority, and fewer cases of so great a disparity between 
aptitude scores and achievement to warrant a check on the test results 
than did the machine. 

Since the pre- interview pupil appraisal was done prior to the 



interview there were no order or interaction effects to be tested. 

2-b. Post- interview pupil appraisal. The hypothesis that there 
would be no significant differences between the automated counseling 
system and the second counselor was accepted for all five of the 
items as is shown in Table 12. 

Question 1. Does the student indicate he is having problems 
with his courses ? 

Question 2. Does the student indicate he is going to college? 

Question 3. Is he planning to take more solids than his GPA 
warrants ? 

Question 4. On the basis of his projected grades and the type 



of college he is planning to attend (or not to attend college or trade 
school at all) is the student's planning realistic, aiming too high, or 



aiming too low ? 
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Question 5. What is the student's proposed college major: 
business, engineering or one of the sciences, social science, language, 
music and art, education, or student is not college bound? 

As shown in Tables 13 and 14, there were no significant order or 
interaction effects for any of the items. 

■ 2-c. Educational decisions. The hypothesis that there would be 
no significant differences between the second counselor and the auto- 
mated counseling system regarding the specific courses selected for 
high school was rejected for the tenth, eleventh, and twelfth, grades 
as can be seen in Table 15. 

Tables 16 and 17 show that the effects of order in which the two 
interviewing modes were presented and the interaction between the 
treatments and order were not significant for any of the itt;ms. 

By taking into consideration the number of courses selected 
during the machine interview, the number selected during the 
counselor's interview, and the number of identical courses selected 
during both interviews, the percentage of agreement was 61 percent 
for the tenth grade, 52 percent for the eleventh grade, and 49 percent 
for the twelfth grade. 

2-d. Completeness of course plans . The hypothesis that there 
would be no significant differences between the automated counseling 
system and the second counselor in terms of students making total. 
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partial, or no course plans was accepted for all three grade levels 
as is shown in Table 18. 

Tables 19 and 20 show that the effects of order and the inter- 
action between treatments and order were not significant* 

Comparison of Group 1 (Model Counselor 
and Machine) with Group 11 (Second 
Counselor and Machine) 

While the automated counseling system was modeled after a 
single counselor's behavior, the third major aspect of the study 
concerned the similarity of his behavior and that of another counselor. 

If. in fact, a second counselor behaved similarly to the model counselor, 
then there is some justification in comparing counselorji with the 
machine system. 

An analysis was performed to determine those validity criteria 
for which the students could be considered as one group in regard to 
human counselors, and those for which the students must be treated 
as two separate groups each in regard to a specific counselor.* 

(Certain analyses done in regard to questions beyond the three 
major hypotheses, and reported in a later section, will combine the two 
human counselor groups. ) 

Hypothesis 3. The extent of agreement between the model 
counselor and the machine will not be significantly different from the 
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TABLE 19 

TABLE OF ORDER F'S FOR COMPLETENESS OF COURSE PLANS COMPARING THE AUTOMATED 

COUNSELING SYSTEM WITH THE SECOND COUNSELOR 



TM-2611/000/00 
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extent of agreement between the automated system and the second 

counselor regarding: 

a. pre- interview pupil appraisal, 

b. post- interview pupil 'ippraisal, 

c. educational decisions, and 

dt completeness of educational plans* 

Since the nature of each validity measure has been reported 
separately in sections comparing the automated system and the model 
counselor, and the automated system and the second counselor, they 
are not repeated here unless .he differences between Group I and Group 
II are significant. 

The results of the analysis done by comparing the deviation 
scores (the differences between the machine and the counselor in 
scoring units) from the model counselor with the deviation scores of 
the second counselor are reported in Table 21. 

The hypothesis that there would be no significant differences 
between Group I and Group II was accepted for all validity measures 
included in: 

3“ a. Pre- interview pupil appraisal , 

3-b. Post- interview pupil appraisal, 

3-c. Educational decisions, and 



3-d. Complett^ness of educational plans. 
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By taking into consideration the total number of courses selected 
for the machine interview, the number of the counselors' interview, 
and the total number of identical courses selected in both interviews, 
the machine showed agreement of 62 percent with the model counselor, 
a.id 55 percent with the second counselor. Although the machine 
matched more closely the model counselor in terms of specific 
courses selected, these differences were not great enough to be 
significant. 

In addition to testing the hypothesis of no significant differences 
between Group I and Group II, the effects of the order of interview 
mode presentation on the total sample was investigated. This was 
done by comparing the deviation scores between the machine and the 
counselors for those students who had the machine interview first with 
those who had the coimselor interview first as shown in Table 22. 

For this combined group there were no significant differences in 
regard to order for any of the validity measures except for tenth grade 
course plans. For tenth grade course plans there was significantly 
greater variance between the counselors and the machine when the 
machine was first than when the counselor was first. 

Thus, for the purpose oi the subsequent analyses reported 
below, the findings just discussed justify considering the total 
sample as one group receiving two treatments; (1) counselor 
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interview and (2) machine interview. In addition, order is disregarded 
except for tenth grade course plans in which order 1 (machine first) is 
treated apart from order 2 (counselor first). 

Further Analyses 

In addition to testing the three major hypotheses, several other 
analyses were done. The first three analyses listed below continue to 
make comparisons between the model counselor and the machine and 
between the second counselor and the machine. The fourth analysis 
treats the entire sample as one group making no distinction between 
the model counselor and the second counselor subgroups. 

1. A comparison between the counselors and the machine using 
the number of students making total course plans as the criterion 
measure. This differs from hypotheses 1-d, 2-d, and 3-d in that only 
the students who make total course plans are compared. This analysis 
determines the percentage of students making total course plans 
whereas the testing of the hypotheses 1-d, 2-d, and 3-d determined 
whether or not there were significant differences regarding students 
making total, partial, or no course plans. 

2. A comparison between the experimentally chosen tenth grade 
course plans (those chosen during the machine and the counselors' 
interview) and the tenth grade courses for which the students officially 
registered two weeks following the collection of data. 










20 August 1965 120 TM -26 11/000/00 

3. A determination of the nature of course changes by comparing 
the courses selected during the counselor's interview with the courses 
selected during the machine interview for all three high school grades, 
the machine chosen tenth grade courses with final tenth grade courses, 
and the counselor chosen tenth grade courses with final tenth grade 
courses. 

4. An analysis of the items which indicated significant differences 
between counselors and the automated counseling system regarding 

sex, national and local School and College Ability Test norms, and 
family socio-economic level. 

In cases where the significance of differences could be tested 
the null hypotheses was used, i. e. , the differences were attributed to 
chance unless they exceeded the . 05 level. Some data are reported in 
simple percentage form. 

Comparison Between Machine and Counselor Using Students 
Makin g Total Course Plans as the Validity Measure. An objective 
for determining the effectiveness of an educational planning inter- 
view might be whether or not the student is able to make complete 
course plans. Disregarding the agreement between the counselor and 
the machine in terms of actual courses selected or the comparison of 
students making total, partial, or no course plans between the counselor 
and the machine, this analysis compares the two counseling modes 

o 






ip 
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in terms of the percentage of studenjts who made total course 
plans . 

The percentages as shown in Tables 23, 24, and 25, have been 
computed for the machine and the model counselor for the tenth, 
eleventh, and twelfth grades, the machine and the second counselor 
for the tenth, eleventh, and twelfth grades, and a total comparison 
between the computer and the counselors combining all three high 
school years. 

Comparison between the Model Counselor and the Computer in 
Terms of Total Course Plans. The percentage of students who made 
total course plans was computed for the model counselor and the 
machine in terms of tenth, eleventh and twelfth grades. An average 
percentage for all three years was computed for the counselor and for 
the machine. 

Using a test to determine the significance of differences between 
uncorrelated percentages indicated that while more students who saw 
the counselor than those who saw the machine made total course plans 
these differences were not significant. The counselor's interview did 
not produce significantly more total course plans than the machine 
interview. 

Comparison Between the Second Counselor and the Computer in 
Terms of Total Course Plans. The percentage of students who made 
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TABLE 23 

COMPUTER VERSUS MODEL COUNSELOR IN TERMS 
OF TOTAL COURSE PLANS 



Course Plans. 


Interview Mode 


Percentage making 
Total Plans 


10th grade 


Machine 


95% 


10th grade 


Counselor 


90% 


1 1th grade 


Machine 


70% 


11th grade 


Counselor 


75% 


1 2th grade 


Machine 


55% 


12th grade 


Counselor 


70% 


Total 


Machine 


73% CR=.38 



Total Counselor 78% P^.05 
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TABLE 24 

COMPUTER VERSUS SECOND COUNSELOR IN TERMS 
OF TOTAL COURSE PLANS 



Course Plans 


Interview Mode 


Percentage making 
Total Plans 


10th grade 


Machine 


94% 




10th grade 


Counselor 


100% 




11th grade 


Machine 


67% 




11th grade 


Counselor 


33% 


• 


12th grade 


Machine 


56% 




12th grade 


Counselor 


22% 




Total 


Machine 


72% 


CR = 1. 67 


Total 


Counselor 


52% 


P ^ .05 


TABLE 25 



COMPUTER VERSUS COUNSELORS IN TERMS 
OF TOTAL COURSE PLANS 







Percentage making 


Course Plans 


Interview Mode 


Total Plans 



Total 



Machine 



73% CR = .39 
69% P .05 



Total 



Counselor 
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total course plans was computed for the second counselor and th<p« 
machine in terms of tenth, eleventh and twelfth grades. An average 
percentage for all three years was computed for the counselor and the 
machine. 

A test of the significance of differences between uncorrelated 
percentages indicated that while on the average more students who saw 
the machine than those who saw the counselor made total course plans 
these differences were not significant. The machine interview did not 
produce significantly more course plans than the counselor's interview* 

Total Comparison Between the Counselors and the Computer in 
Terms of Total Course Plans. An average percentage for all three 
years of high school course plans and for both counselors was compared 
with the computer's average total course plan percentage for the three 
high school years. 

While a slightly larger percentage of students made total course 
plans during the automated interview the differences between the per- 
centage cf students making total course plans during the automated 
interview and the percentage of students making total course plans 
during the counselors' interview were not significant. 

Comparison Between Experimentally Chosen Tenth Grade Course 
Plans and Actual Tenth Grade Course Plans. Shortly after the data for 



this study were collected the students registered for their actual tenth 
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grade courses. Although the prime objective of this study was to 
determine the differences between the computer chosen and the 
counselor chosen course programs another facet which merited 
investigation was the relationship between the experimentally chosen 
course programs, the model and second counselor and the machine 
interviews, and the courses actually selected by the student for the 
tenth grade. 

Comparisons were made between the machine interview- order 1, 
having the machine interview first, and actual tenth grade plans; 
machine interview- order 2, having the counselor s interview first, and 
actual tenth grade plans; counselor’s interview- order 1 and tenth grade 
plans; and counselor's interview -order 2 with actual tenth grade plans. 
Table 26 reports the results of this analysis. 

Comparison Between Machine Chosen Tenth Grade Course Plans 

and Actual Tenth Grade Course Plans. 

1. Machine Interview -- Order 1 

The course plans for the tenth grade made during the 
automated system interview for those students seeing the machine 
first, subgroups lA and IIA, were compared with the actual tenth 
grade course plans for these same students. 

The hypothesis that there would only be chance differences 
between the machine interview course plans and the actual course 
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significant beyond .05 
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pleins was rejected. There were significant differences between 
the machine course plans and the actual course plans. 

2. Machine Interview -- Order 2 

The course plans for the tenth grade made during the auto- 
mated system interview for those students seeing the counselor 
first, subgroups IB and IIB, were compared with the actual tenth 
grade course plans for these same students. 

The hypothesis that there would only be chance differences 
between the machine interview course plans and the actual 
course plans was rejected. There were significant differences 
between the machine course plans and the actual course plans. 

3. Comparison of the Machine 1st and Counselor 1st Groups 
in Terms of Machine Course Plans Versus Actual Course Plans. 

In an attempt to determine if there were any differences 
between the machine 1st (order 1) and counselor 1st (order 2) 
in making the comparison between machine tenth grade course 
plans and actual course plans the deviatiop. scores between the 
machine-order 1 group was compared with the deviation scores 
of machine- order 2. A deviation score was computed for each 
student by scoring the difference between the machine chosen 
course plan and the actual course plan. 
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The hypothesis that there would be only chance differences 
between machine- order 1 and machine-order 2 was accepted. 

The differences were not significant. 

Comparison Between Counselor Chosen Tenth Grade Course 
Plans and Actual Tenth Grade Course Plans. 

1. Counselors' Interview -- Order 1 

The course plans for the tenth grade made during the 
counselors' interview for those students seeing the machine 
first, subgroups LA and IIA, were compared with the actual 
tenth grade course plans for those same students. 

The hypothesis that there would only be chance differences 
between the counselor's interview course plans and the actual 
course plans was accepted. There were no significant differences 
between counselors' interview^ order 1 and the actual tenth grade 
course plans. In making their actual course plans the students 
follow closely the plans made during the counselors' interview. 

2. Counselors' Interview -- Order 2 

The course plans for the tenth grade made during the 
counselors' interview for those students seeing the counselor 
first, subgroups IB and IIB, were compared with the actual tenth 
grade course plans for these students. 
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The hypothesis that there would be only chance differences 
between the counselors' interview course plans and the actual 
course plans was accepted. There were no significant differences 
between counselors' interview-order 2 and the actual tenth grade 
course plans. In making their actual course plans the students 
follow closely the plans made during the counselors' interview. 

3. Comparison of Machine 1st and Counselor 1st Groups in 
Terms of Counselors' Course Plans Versus Actual Course Plans. 

In an attempt to determine if there were any differences 
between machine 1st (order 1) and counselor 1st (order 2) in 
making comparison between counselors' tenth grade course plans 
and actual course plans the deviation scores between the 
counselors'-order 1 group was compared with the deviation 
scores of counselors'-order 2. A deviation score was computed 
for each student by scoring the differences between counselor 
chosen course plan and the actual course plan. 

The hypothesis that there would be only chance differences 
between counselors'-order 1 and counselors'-order 2 was 
accepted. In terms of deviation from final course plans there 
were no significant differences between the counseling interview 



being first or last. 
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Nature of Course Changes. The nature of course changes 
involved two aspects; from what kind of course to what kind of course 
did the students move, and was the movement out of an academic area 
or did it remain in the same area. The nature of change has been 
investigated for counselor- machine differences, machine- final course 
plan, differences, and counselor- final course plan differences. 

Nature of Course Changes Between the Machine Interview and 
Covoiselors' Interview. Table 27 shows that three types of movement 
were responsible for 68 percent of the course changes. These types were 
from special unit to special unit (32%), elective to an elective (22%), 
and from elective to no course (14%). Each of the remaining twelve 
possible types of changes were responsible for 0 to 5 percent of the 
change. 

Of these changes, in the tenth grade 68 percent involved changing 
from one course to another in the same field, for example, a change 
from art history to ceramics, in the eleventh grade 50 percent of the 
changes were within the same field and in the twelfth grade 45 percent. 
A total of 93 course changes were made. 

Nature of Course Changes Between Machine Interview and 
Actual Tenth Grade Course Plans. Four types of movement were 
responsible for 71 percent of the course changes as can be seen in 
Table 28. These types were from special unit to elective (17%), 
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TABLE 27 



NATURE OF COUNSELOR AND MACHINE COURSE CHANGES 



Direction of Change 


10 th 
grade 


Nuniber of Changes 

nth 12th 

grade grade Total 


% of 
Change 


Special Unit to Special Unit 


6 


13 


10 


29 


32% 


Special Unit to solids 


2 


0 


1 


3 


3 

1“ 


Special Unit to elective 


2 


2 


1 


5 


5 

A 


Special Unit to no course 


0 


1 


3 


4 


4 


Solid to Special Unit 


1 


1 


0 


2 


2 


Solid to solid 


1 


0 


1 


2 


2 


Solid to elective 


0 


0 


0 


0 


0 


Solid to no course 


0 


1 


1 


2 


2 


Elective to Special Unit 


2 


3 


0 


5 


5 


Elective to solid 


0 


0 


0 


0 


0 


Elective to elective 


7 


7 


6 


20 


22 


Elective to no course 


5 


3 


5 


13 


14 


No course to Special Unit 


0 


0 


3 


3 


3 


No course to solid 


1 


1 


0 


2 


2 


No course to elective 


0 


2 


1 


3 


3 


Total 


27 


34 


32 


93 


99% 
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TABLE 28 

NATURE OF COURSE CHANGES BETWEEN MACHINE AND 

FINAL lOTH GRADE PLANS 



Direction of Chanf 



Special Unit 

Special Unit 

Special Unit 

Special Unit 

Solid 

Solid 

Solid 

Elective 

Elective 

Elective 

No course 



to Special Unit 
to solids 
to elective 
to no course 
to Special Unit 
to solid 
to no course 
to Special Unit 
to elective 
to no course 
to elective 



% of 
Change 

4 
4 
17 
4 
4 
4 
4 
4 
21 
8 
25 



Number of 
Cha^iges 

1 
1 

4 
1 
1 
1 
1 
1 

5 
2 

6 



Total 



99 



24 
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elective to elective (21%), elective to no course (8%), and no course to 
elective (25%). Each of the remaining twelve possible types of changes 
were responsible for 0 to 4 percent of the change. 

Of the changes between the machine course plans and final course 
plans 43 percent of the changes were within the same area. There was 
a total of 24 course changes. 

Nature of Course Changes Between Counselors' Interview and 
Actual Tenth Grade Course Plans. Only three types of movement out 
of fifteen possible types were present as indicated by Table 29. 

Special unit to elective (57%), elective to elective (14%), and no course 
to elective (29%), were the types involved. 

Only 20 percent of the changes in courses were in the same field. 
And there were only seven changes between the counselors' interview 
and the final course plans. 

Analysis Based on Sex . The analysis using sex as a variable 
tested the differences between male and female in terms of deviation 
scores. The deviation scores were compu^d by scoring the differences 
between the counselors' interview and the automated counseling system 
interview. The group was composed of 23 males and 17 females. 

Previous analyses indicated no significant differences between 
order 1 (machine 1st) and order 2 (counselor 1st) except for tenth 
grade course plans. Therefore the order 1 and order 2 machine 
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TABLE 29 

NATURE OF COURSE CHANGES BETWEEN COUNSELOR AND 

FINAL lOTH GRADE PLANS 



Direction of Change 


% of 
Change 


Number of 
Changes 


Special Unit to elective 


57 


4 


Elective to elective 


14 


1 


No course to elective 


29 


2 


Total 


100 


7 
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groups were combined into one group and the order 1 and order 2 
counselor groups were combined into one group except for tenth grade 
course plans which was analyzed by order 1 and by order 2. For this 
analysis both counselors were combined into one group and all machine 
interviews into one group« 

The items which were analyzed and reported in Table 30, 
those which showed significant differences in previous analyses testing 
the three major hypotheses were; GPA, pre-interview appraisal 
statements 4, 5, 6, 7, 13, post- interview appraisal question 6, 
course schedule tenth grade-order 1, course schedule tenth grade- 
order 2, eleventh grade course schedule, twelfth grade course schedule, 
and completeness of course plans for the twelfth grade. 

The resulting analysis showed no significant differences between 
male and female on any of the items. Sex did not cause any of the 
differences found between the machine and counselor outcomes. 

Analysis Based on School and College Ability Test Scores. Two 
separate analyses were done. One using national SCAT norms and 
one using local SCAT norms. In both cases the students were divided 
into two groups: the high SCAT group (those students above the norm 
mean) and the low SCAT group (those students below the norm mean). 
The mean for the local norm was 65 percentile and the mean for the 
national norms was 50 percentile. The scores ranged from 3 to 97 



20 August 1965 



136 



TM-2611/000/00 



w 



w 

i/> 



U) 

w 

tf) 

Wl 

(i6 

bu 

Q 

PC 

o 

bu 

bb 

a 



'd 

|8 



C A 
P- 

P 



Ui 

>S 

«l 

a 



CS M 

.5 a 

Si 

So 



CS M 

•Sa 
.5 ? 

|s 



u> 






li) 

8 



4 



00 

CO 

CO 



in 



CO 


CM 


yo 


00 X 




8 


8 


CM 

d* 


^ 1 


1 8 



CO 



(S 


CM 


CM 


CM 


CM 


CM 




o\ 




00 


.in 


CM 




















CM 


CO 




• 

























00 

lO 




CM 


8 


8 


0\ 


CO 

CTj 


lO 




CM 




CM 


CM 






• 


• 


• 






• 


CO 


in 






00 


00 


o 


in 




Q 


lO 


00 








o 


O 




o 




















CO 


Cs 


o\ 


CO 


CO 


CM 


in 


CO 


o\ 


a% 


CO 


00 


in 


CO 


O 




CO 


00 


00 








• 


00 


id 


as 


OS 


















CO 


in 






00 


00 


o 


in 


Cv 


Q 


lO 


00 








o 


O 




o 




















CO 


00 


00 


00 


00 


CO 




CO 


CO 


CO 


CO 


CO 


CO 


CO 



CO 

CO 



CO 

8 



00 

00 



in 

CM 

in 



in 






00 


lO 


in 




o 


in 


in 


C\ 


CM 


CO 


O 


o 



CM 


CM 




o 






o\ 


o\ 


o 




o\ 




o 




lO 


• 


• 


% 


• 


• 




00 


CO 


M* 


in 


CM 




r>l 




CO 


00 


lO 


in 




8 


8 


CO 

CO 


in 

o 


o\ 

o 



lO 



00 



CO 

CM 



8 



lO 

CO 



g 

r 

T3 



C 

'S 



01 

00 

rt 

0 

1 

I 13 
0)"’ 
bou 

o 01 

o'd 

, ™ 



d 5^ 

‘fl tSrt 



0 

T3 

01 



T3 

0> 

liS 

0< 



a 

01 

S 

cu 

d 



a 

0) 

*3 

uS 



*da 

(y*M 

■Sir 
« 01 
JS'O 

01 

•go 

ii 

in 01 



in 



c 

01 

u 

d 

g; 

S- 

U) 

6 

01 

I 

a 



*4 

.2 

o. 

.r 



0 

u 

1 
1^ 



01 

a 

0 

ca • 

Z Ul 
If Ml 

«s 

mU 

01 ^ 
*2 01 

SI 

•i 

ti d 

^ts 

fiS 

fio 



a 

01 

§. 
>4^ 
Al VI 

.S 01 

d-S 

•S-tt 

o« 

M rrt 
01 

M 4-* 

o u 

U <P 

V) Q. 

'Q V) 

Is 

4>J 

Old 
d 01 



0 I*A 4 

<U 



<V* 

L« 

0 jj 

6 

0» 

00 

01 

0 
u 

T3 

01 

vs 

I 



- « 



3 

01 

§ 

u 

01 

•S 

h 

bO 



01 

■2 

bO 

rS 



bOg 

“J 

V4 
01 ^ 
II 

in 01 



> W • 

£ a M 


a 








•dJo 

w d M 


J/S 

3 


01 

9 


01 

9 


01 

9 




01 

'S 


T3 

01 


9 


T3 

01 




1 


•s 


•s 


•s 


S*«>'S 


d 


ui 


VI 


VI 




01 

VI 

S 


01 

VI 

3 


01 

B 




•C 


o 


o 


o 


H»U 




U 


U 


U 



01 

•s 

»-t 

00 

•s 

CM 

01 

*3 

T 3 

0) 

•s 

V) 

01 

c/s 

I 

u 



01 

d 

& 

•5 

CM 



3 

d 



01 

§ 

u 



13 

01 

a 

01 



a 

i 

u 



CO 



in 



er|c 



20 August 1965 



137 



TM-2611/000/00 



percentile. Using the national norms there were 28 students in the 
high group and 12 students in the low group. Using the local norms 
there were 24 students in the high group and 16 students in the low 

group. 

Previous analyses indicated no significant differences between 
order 1 (machine 1st) and order 2 (counselor 1st) except fpr tenth 
grade course plans. Therefore the order 1 and order 2 machine 
groups were combined into one group and the order 1 and order 2 
counselor groups were combined into one group except for tenth grade 
course plans which was analyzed by order 1 and order 2. For the 
entire analysis both counselors were combined into one group and all 

machine interviews into one group. 

The items which were analyzed and reported in Tables 31 and 

32, were; GPA, pre-interview appraisal statements 4, 5, 6, 7, 13, 
post- interview appraisal question 6. course schedule tenth grade-order 
1. course schedule tenth grade-order 2. eleventh grade course 
schedule, twelfth grade course schedule, and completeness of course 

plans for the twelfth grade. 

Results Based on National SCAT Norms. The items which showed 
significant differences between high and low groups were GPA, 
appraisal statement 4 -- Student is a potential drop-out, appraisal 
statement 7 -- Student is getting better grades than one would expect 
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from looking at aptitude scores, and completeness of twelfth grade 
course plans. In all cases where significant differences occurred there 
was significantly less variance with the high group than with the low 
group. 

Results Based on Local SCAT Norms. Items which shewed 
significant differences between high and low groups were appraisal 
statement 4 -- Student is a potential drop-out and appraisal statement 
7 -- Student is getting better grades than one would expect from looking 
at aptitude scores. In each of the above cases there was significantly 
less variance with the high SCAT group than with the low group. 

Analysis Based on Socio-Economic Level . Students were divided 
into three socio-economic groups based on their father's or mother's 
(if father was deceased) occupation: professional and managerial, 
semi-professional, and labor including skilled, semi-skilled, and un- 
skilled labor. When the vocational classification, a modification of 
Roe's occupational classification system, was in question the education- 
al level was used as a determinant. College or college plus graduate 
work was used for the profession- managerial group, high school or 
high school and some college was used for the semi-professional 
catagory, and less than high school was used for the labor classification. 
There were 20 students in the professional- managerial group, 9 
students in the semi-professional group, and 11 in the labor group. 
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Previous analyses indicated no significant differences betv/een 
order 1 (machine 1st) and order 2 (counselor 1st) except for tenth 
grade course plans. Therefore the order 1 and order 2 machine 
groups were combined into one group and the order 1 and order 2 
counselor groups were combined into one group except for tenth grade 
course plans which w»as analyzed by order 1 and order 2. For the 
entire analysis both counselors were combined into one group and all 
machine interviews into one group. 

The items which were analyzed and reported in Table 33 were 
GPA, pre- interview appraisal statements 4, 5, 6, 7, 13, post-inter- 
view appraisal question 6, course schedule tenth grade order 1, course 
schedule tenth grade order 2, eleventh grade course schedule, twelfth 
grade course schedule, and completeness of course plans for the 
twelfth grade. 

The resulting analysis showed no significant differences between 
the three socio-economic groups on any of the items. Socio-economic 
level was not responsible for any of the differences found between 
machine and counselor outcomes. 

Analysis of Specific Limitations of the 
Computer Based Counseling System 

Since it was possible that some of the qualities found normally 
in a human counselor's interview might be lacking in the automated 



TABLE 33 

TABLE OF F'S FOR DIFFERENCES BETWEEN SOCIO-ECONOMIC LEVELS 
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counseling interview the students were asked questions concerning 
machine limitations, attention span, reinforcers, sensitivity to need 
an4 orientation, student cooperation, and reservations about course 
plans during a short structured interview following completion of both 
the machine and the counselor's interview. The results of the structured 

interview follows below. 

Machine l.imitations. Of the students who could not make a total 
course plan for the tenth grade none felt that this was caused by the 
machine not presenting the material clearly enough. None of the 
students who failed to make a tenth grade plan attributed this to the 

m* chine lacking necessary information. 

Attention Span. Only one student out of forty terminated the 
machine interview before attempting a course plan. Ninety-four per- 
cent of the students felt the machine interview did not make them feel 

bored or restless. 

Lack of Reinforcers . Twenty- six percent of the students felt 
the fact that they didn't always know whether their responses to the 
machine were appropriate or not increased the difficulty of making 

course selections. 

Of those students not completing a course program twenty per- 
cent felt not having the counselor in the room made it more difficult 
to make adequate course selections. 
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Lack of Sensitivity to Needs and Orientations. Fifty- three 
percent of the students felt the machine was not able to take into 
consideration all the information necessary to meet their personal 

needs in terms of what should be involved with high school course 
planning. 

Lack of Student Cooperation . None of the students tried to jam 
the machine or type in ridiculous responses. 

Reservations About Course Plans . Fifty- six percent of the 
students making tenth grade course plans on the computer had 
reservations about these course plans while only twenty percent of the 
students had reservations about the tenth grade course plans made 



with the counselors. 
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CHAPTER V 

SUMMARY AND CONCLUSIONS 



Summary 



Statement of the Problem 

The problem with which this study has been concerned is the 
validity of an existing computer-based counseling system, which 
was developed to simulate the decision-making behavior and logic of a 
counselor counseling with ninth grade students in regard to their 

educational planning for high school. 

In addition to determining the validity of a computer-based 
counseling system by comparisons between the automated system and 
the model counselor, comparisons were made between a second 
counselor and the automated counseling system. The study was 
concerned with the following two-fold problem. 

1. How well does the automated counseling system agree with 

the counselors in terms of pupil appraisal? 

2. How well does the automated counseling system agree with 
the counselors in educational planning for high school ? 
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Research Methods and Procedures 

Hypotheses . 1. The automated counseling system will not be 
significantly different from the model counselor regarding: 

a. pre- interview pupil appraisal, 

b. post- interview pupil appraisal, 

c. educational decisions, and 

d. completeness of educational plans. 

2. The automated counseling system will not be significantly 
different from the second counselor regarding; 

a. pre- interview pupil appraisal, 

b. post- interview pupil appraisal, 

c. educational decisions, and 

d. completeness of educational plans. 

3. The extent of agreement between the automated system and 
the model counselor will not be significantly different from the extent 
of agreement between the automated system and the second counselor 
regarding; 

a. pre- interview pupil appraisal, 

b. post- interview pupil appraisal, 

c. educational decisions, and 

d. completeness of course plans. 
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Sample. Forty ninth grade students (23 boys and 17 girls) were 
drawn at random from iVilbur Junior High School in Palo Alto, Calif- 
ornia. The students ranged in academic ability, from 3 percentile to 
98 percentile, with a sample mean of 65 percentile, based on School 
and College Ability Test scores, national norms. 

Treatment. The sample was randomly divided into two groups 
of 20 students each. One group was interviewed by the model counselor 
and the automated counseling system, and the second group was inter- 
viewed by the second counselor and the automated counseling system. 

As a precaution to offset the effects of order of interviewing mode, 
each of the two groups was subdivided into two subgroups. One sub- 
group was interviewed first by the machine and then by a counselor, 
and the second subgroup was interviewed first by a counselor and then 
by the machine. 

Nature of the Data Collected . Statements and answers to questions 
relating to pre- interview and post- interview appraisal were obtained 
from the model counselor, the second counselor, and the machine. 

In addition, results of the educational planning interview, that is, the 
specific courses selected for high school and the degree of completeness 
of course planning, and data relating to limitations of the computer- 
based counseling system were collected for each interview. 
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Data regarding student's sex, SCAT scores, family socio- 
economic level and final tenth grade course plans (the courses for 
which the student officially registered two weeks following the 
collection of data) were also collected for subsequent analyses of 
items which proved to have significant differences between the counselor 
and the machine. 

Data Collection Procedures .. Predictions and evaluations about 
the students made prior to the counseling interview were collected 
from the counselors by the use of questionnaires for the purpose of 
pre- interview pupil appraisal. Machine print- outs containing similar 
data for each student were collected. Following the interview, post- 
interview pupil appraisal data (evaluations made on the basis of inter- 
view outcomes) were collected in the same manner as pre-interview 
pupil appraisal data. For the information relating to course planning, 
the student's course schediile form completed during the counselors' 
interview, and the machine output (course selections typed into the 
machine by students), were c ollected. Data involving limitations of 
a computer-based counseling system wei e gathered from the machine 
output and from a structured interview following the completion of 
both interviews. Information such as sex, SCAT scores, family 
socio-economic level, and final tenth grade course plans were taken 



from school files. 
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Data Treatment Methods . A determination of the extent of 
agreement between treatments (machine versus counselor) was the 
prime objective. However, it was necessary to determine what effects 
order of interview mode presentation and interaction between treat- 
ments and order, had upon treatments. The extent of agreement 
between the model counselor and the automated system, arid the extent 
of agreement between the second counselor and the automated system, 
was determined by using a Type I design, which yields F ratios by 
which the significance of treatments, order, and interaction were 
determined. In order to improve the preciseness of the evaluations 
involving course plans a percentage of agreement between the two 
counseling modes were calculated. 

The similarity between two counseling modes or between groups 
was assumed if the F ratio was smaller than that required for the . 05 
level of significance. 

The effects of order on the total sample and the differences 
between Group I (those students seen by the model counselor and the 
machine) and Group II (those seen by the second counselor and the 
machine) were determined by employing a simple randomized design 
utilizing deviation scores (the scored differences between the counselor 
and the machine) for the basis of comparison. 
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The automated counseling system and the counselors were 
compared on the basis of the percentage of students making total 
course plans. 

The extent of agreement between the experimentally chosen tenth 
grade course plans and the final tenth grade course plans were determin-f 
ed by using a simple randomized design. 

The nature of course changes was investigated by determining 
the direction of movement and the percentages of changes involving 
courses in the same adacemic field. 

Subsequent analyses were done for the items which showed 
significant differences between the counselors and the automated 
counseling system. The factors investigated in these analyses were 
sex, scholastic ability as measured by SCAT scores, and socio- 
economic level all involving the use of a simple randomized design. 

An indication of some limitations of a computer-based counseling 
system was determined by student responses during the structured 
interview. These responses were reported on a percentage basis. 

Analyses of Data 

Three major hypotheses were tested: (1) the automated coun- 
seling system will not be significantly different from the model 
counselor, (2) the automated counseling system will not be significantly 



20 August 1965 



151 



TM-2611/000/00 



different from the second counselor, and (3) the extent of agreement 
between the automated system and the model counselor will not be 
significantly different from the extent of agreement between the auto- 
mated system and the second counselor, each in regard to (a) pre-inter- 
view pupil appraisal, (b) post- interview pupil appraisal, (c) educational 
decisions, and (d) completeness of educational plans. 

The Model Counselor and the Automated Counseling System. 
Hypothesis 1 was rejected for the following pre- interview pupil 
appraisal items: predicted GPA; Statement 4. Student is a potential 
drop-out; Statement 5. Should be headed for college-encourage student 
to explore widely in academic areas; Statement 7. Student is getting 
better grades than one would predict from looking at aptitude scores; 
Statement 13. The disparity between aptitude scores and achievement 
is so great that one is led to suspect aptitude test-check test results. 
The hypothesis was rejected for post- interview pupil appraisal 
Question 5, which stated the student's proposed college major. 
Hypothesis 1 was rejected for grades ten and twelve regarding 
educational decisions, and for grade twelve concerning completeness 
of educational plans. 

There were significiant order effects on one item, post- interview 
pupil appraisal Question 1. Does the student indicate he is having 
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problems with his courses, and interaction effects regarding tenth 
and twelfth grade course plans. 

The Second Counseloi: and the Automated Counseling System . 
Hypothesis 2 was rejected for the following pre- interview pupil 
appraisal item: Statement 4. Student is a potential drop-out; Statement 
6. Low counseling priority - no problems apparent; and Statement 13. 
The disparity between aptitude scores and achievement is so great 
that one is led to suspect aptitude test - check test results. Hypothesis 
2 was also rejected for grades ten, eleven, and twelve regarding 
educational decisions. 

None of the items showed either significant order or interaction 
effects. 

Comparison between Group I (model counselor versus the 
automated counseling system) and Group II (second counselor versus 
the automated counseling system). In regard to treatment (counselor 
versus machine) Hypothesis 3’was accepted for all items. There were 
no significant differences between the two counselors. In determining 
the total effects of order, combining the results of both counselors, 
there v/as a significant difference for only the tenth grade course plans. 
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Further Analyses 

Cornpetrison Between Counselors and the Automated Counseling 
System in Terms of Total Course Plans. There were no significant 
differences between the counselors and the machine in terms of the 
percentage of students making total course plans. 

Comparison Between Experimentally Chosen Tenth Grade Course 
Plans andFinal Tenth Grade Course Plans. There were significant 
differences between computer chosen tenth grade courses and final 
tenth grade course plans (those courses for which the students officially 
registered two weeks following data collection), but there were no 
significant differences between counselor selected tenth grade courses 

O 

and final tenth grade course plans. 

Nature of Course Changes. Most of the movement involved three 
types of changes, special unit to special unit, elective to elective, and 
elective to no course. Over 50 percent of the changes involved moving 
from one course to another within the same academic field. 

Analyses Based on Sex, SCAT Scores, and Family Socio-economic 
Level. Of the items which showed significant differences between the 
counselors and the computer none indicated the differences were due 
to sex. When analysis was done using SCAT scores -- national norms 
as the basis for comparison, there were significant differences 
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between hign and low groups on predicted GPA, pre- interview pupil 
appraisal statement 4, pre- interview pupil appraisal statement 7, 
and completeness of twelfth grade course plans. When using SCAT 
scores, local norms, significant differences were found for appraisal 
statements 4 and 7. Analyzing the items in terms of family socio- 
economic level showed no significant differences between any of the 
levels. 

Limitations of a Computer-based Counseling System « The 
majority of the students felt the presentation of the mateiial, the 
nature of the material, and the amount of machine reinforcement or 
feedback was sufficient to make course plans. However, 53 percent 
felt the machine could not take into consideration all the information 
necessary to meet their personal needs in terms of adequate course 
planning. Fifty- six percent of the students had reservations about the 
tenth grade course plans chosen by the aid of the machine while only 
20 percent had such reservations about plans made with the aid of 
the counselor. 

Discussion and Conclusions 

Discussion 

The major implication of this study can be identified within 
the following three general catagories: 



20 August 1965 



155 



TM- 26 11/000/00 






(1) implications for further research and development of the 

automated counseling system itself, 

(2) implications for experimental implementation of the auto- 
mated counseling system in a real school environment and, 

(3) implicationsfor other related research and development 
concerned with automation in counseling and guidance. 

As will become obvious in the further discussion, these are not 
mutually exclusive catagories. In spite of the overlap, they do provide 
a useful vehicle for discussion the major implications of this study. 

Implication for Further Research and Developm ent of the Auto- 
mated Counseling System. The question arose during the pilot phase 
of this study as to whether a composite counseling model should be 
built, which would incorporate the best features of several counselors, 
or whether the behavior of one counselor should be simulated. The 
problem of establishing validity criteria for the simulation of a model 
counselor was a less difficult one than for a composite system. How- 
ever, it could be possible to build a composite counseling model if 
the differences between the model counselor and a second counselor, 
regarding the criteria used in this study, were not significant even 
though they were different regarding counseling style. The results 
of this study support the notion that a working composite counseling 
model can be built. In analyzing the interview tape recordings of many 
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counselors during the pilot study, it was apparent that a great deal of 
variance existed regarding counseling style. The model counselor's 
style was very straight forward and the counseling process was not 
difficult to flow chart. The second counselor's style while involving 
identifiable rules had a great deal of variance concerning these. This 
would have necessitated the analysis of many more interviews before 
adequate criteria for the use of these rules could be established. Yet 
in terms of counseling outcomes, based on the criteria used in this 
study, there were no significant differences between the two counselors. 

In terms of further research, it must be determined how much 
non** actuarial data should be included in the machine program regard- 
ing pupil appraisal. It may be that the use of a good deal of non- 
actuarial data would not improve the predictive validity of the machine 
pupil appraisal. The second counselor using, in addition to actuarial 
data, extensive personal knowledge of the students had closer agree- 
ment with the machine than did the model counselor using actuarial, 
cumulative record, and interview data. If the inclusion of a large 
amount of personal information about the students, does improve 
prediction, then the rules for the use of this data could be identified 
and incorporated into the program. Even if the inclusion of personal 
student information does not improve the predictive validity of factors 
related to pupil appraisal, this information could be used in further 
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application of an automated system in other areas of counseling such 
as, educational- vocational planning and personal problems. 

Though the differences between the machine and the model 
counselor were generally significant regarding the selection of 
specific courses, 66 percent of agreement for the tenth grade is a 
fairly acceptable rate for the machine considering this was the first 
run. With the possibility of improving the percentage of agreement 
(discussed in the section dealing with the implementation of the 
automated counseling system into a real school environment) it 
appears that it would be possible to simulate a counselor aiding 
students in other decision-making tasks. The next step would be to 
define the tasks to be simulated and attempt to identify the rules used 
in this process. 

The failure of the machine system to achieve a greater degree 
of simulation in regard to course selection may be attributed to 
several causes. It is possible that the computer program did not 
include critical rules used by the counselor. On the other hand, 
the program may be adequate, but the interaction with the machine 
itself might constitute a situation including certain aspects (unrelated 
to the program) which have an adverse affect on the pupils course 
selection behavior, in that the program inadvertantly included rules 
or at least features which were unlike the behavior of the model 
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counselor. It could, of course, be a combination of all three. It 
may also be that there are certain counselor qualities which will 
defy programming, such as sensitivity to non-verbal cues. 

Part of the feasibility of using an automated counseling system 
in schools was supported by the students* enthusiastic acceptance of 
the system. A question arises regarding the use of such a system 
after implementation into a school system. If the automated system 
is seen as just a novelty, the use of the machine will diminish rapidly. 
However, if tne same pattern is followed concerning the use of the 
system as with other data systems, the problem will become that of 
over use and pressure tO' expand the system to do more and more 
things rather than disuse. 

This study has been concerned with the simulation of a single 
counselor. No attempt has been made to determine which of the two 
counseling modes is better in terms of course planning or pupil 
appraisal. A machine model designed to better a counselor could be 
investigated. Instead of simulating a single counselor, or a composite 
of several counselors, it might be possible to establish criteria for 
a super counseling model which would be superior to any single 
counselor or composite of counselors. It may be that such a model 
would be too good. Some of the student acceptance of the automated 
counseling system may have been related to the fact that it attempted 
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to simulate a human counselor, not only regarding counseling rules 
but also regarding counseling style. A super model might be rejected 
by the students since in developing its superiority some of the 
desirable human traits may have to be de- emphasized. 

Implication for Experimental Implementation of an Automated 
Counseling System in a Real School Environment. Regarding the present 
computer-based counseling program certain, modifications will have 
to be made before the greatest use can be made of it in a real school 
environment. 

1. The pre- interview pupil appraisal items which indicated 
significant difference between the counselor and the machine were 
related to academic ability. Analysis showed there was agreement 
between the machine and the counselor pertaining to the high SCAT 
score group but not the low score group. The lower end of the 
machine's prediction curve will have to be elevated to match that of 
the counselor. 

2. Safeguards against students taking courses out of sequence 
or without proper prerequisites will have to be built into the program 
since 50 percent of the course changes involved courses in the same 
academic areas. 

3. The courses will have to be separated into a number of 
catagories based on academic difficulty. The student will be asked 



20 August 1965 



160 



■EM-2611/000/00 



to look at the courses in his ability range before choosing courses 
above or below his ability. The present program grouped courses 
into special units, solids and electives. But some solids are more 
difficult than some special units and some are easier than certain 
electives. 

Factors involving student's interests and personality should 
be included in the revised program. 

5. A short course description defining the nature, the academic 
level, and necessary prerequisites for each course will have to be 
available to the student. 

The implementation of the automated counseling system would 
have two major functions, (1) that of a computer based pupil informa- 
tion system and (2) counselor aid regarding educational planning. 

The current machine program, with the minor modification 
already suggested, would be a valuable computer-based pupil informa- 
tion system. It would have value to both the students and the counselor. 
The students could use the system as a means of getting up-to-date 
progress reports on themselves, predictions about future grades, 
colleges available to them, and answers to questions regarding the 
nature of specific high school courses. There are systems today 
which use diagnostic data but unlike the proposed system students do 
not have direct accessability to it. The counselor could use the 
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system by getting a print- out prior to each educational planning inter- 
view containing actuarial student appraisal data. Problem areas 
could be identified by the machine to be worked on during the counseling 
interview. The current systenci would have to be used as a counselor's 
tool in educational planning. The students could have ready access to 
the machine when they wished to make course plans. Out of 300 
students only 150 may choose to even attempt a course planning inter- 
view by the machine. The counselor would review the course selection 
made by those students. Out of the 150 students who made machine 
course plans, the counselor may call in 50 students for purposes of 
discussing the possibility of revising their machine selected course plans. 

Implications for Other Related Research and Development 
Concerned with Automation in Counseling and Guidance. Although it 
may be threatening to consider use of automation in an endeavor 
which relies so much on human characteristics, the rapidly expanding 
volume of information, with the accompanying need for students to 
have and use wisely, makes it necessary to give serious thought to 
automation in counseling and guidance. It is not suggested that 
machiz^es replace counselors but that man- machine combinations, 
utilizing the favorable characteristics of both, might be used to meet 
the counseling and guidance needs of current and future generations. 
Further investigation of the counseling process through the systems 
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approach may uncover better methods and criteria for the training of 
counselors. Some of the problems relating to counseling and guidance, 
and the application of computerized system in these areas, are dis- 
cussed below. 

Human counselors have certain qualities such as acceptance, 
empathy, and sensitivity to non-verbal communication. It is difficult 
to teach counselors' in training these qualities. It is often assumed 
that a person either has, or doesn't have, these traits and while train- 
ing may improve these qualities, it doesn't create them. It may be 
possible, through analysis of both audio and video tape recordings of 
expert counselors' interview, to give behavioral definition of acceptance, 
empathy, and sensitivity. While these traits may never be programmed 
into a machine, if they are definible in behavioral terms, then per- 
haps some criteria can be developed which would snable these qualities 
to be taught to counseling students. 

Another problem of an automated counseling system is that of 
interface, communication directly from the machine to a human. 
Although the students did not indicate any adverse reaction to the 
machine regarding interface, it is not known whether human to human 
interface was simulated in this study. The question arises as to 
whether a machine can simulate human to human interface. A 
problem also in machine-human interface is that of the communication 
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stimulus. Would verbal communication be more effective than written 
communication ? 

Conclusions 

On the basis of the results reported and the discussion in this 
chapter it is concluded that: 

(1) The development of a composite counseling model which 
would incorporat the best features of several counselors is possible, 

(2) the current automated counseling system, with the 
modifications discussed in this chapter, has value as a computer based 
pupil information system, and has use as an educational planning aid 
when used in conjunction with a counselor, and 

(3) most students would voluntarily use the automated counseling 
system if it were implemented into a school system. 
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APPENDIX A 

PROGRAM FOR AUTOMATED INTERVIEW 



As a result of the final phase of the preceding rapport section 
the student is asked to enter his code number which will have been 
assigned previously. 

!• Enter Code Number - - Begin Interview 

Machine validation of code number. Search out the student's name 
in stored data and write out: 

Is your name ? 

A. Yes 

B. No 

Please respond. 

If A. -- proceed 
If B. -- print out: 

Please raise your hand and you will receive further 
instructions. 

Your record indicates that as of the end of the last grade period, 
you were taking the following courses and earning the grades 
indicated for each course: 



A. 

B. 

C. 

D. 

E. 



II. Determination of Problem Areas 

Do you find that you are having any problems with any of these 
courses: 
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A. Yes 

B. No 

If A. -- write out: 

Please refer the above list of courses and enter the 
letter(s) of the course(s) with which you are having 
problems. 

store responses for counselor write out with lead line: 

This student indicates that he is having problems with 
Course 

-- store S response to be used at Section HI. 



-- proceed to Section III Nature of Problem. 

If B. -- Compare grades with S expectancy, 
expectancy 

proceed to Section IV Post High ochool Plans. 



expectancy 

Write out for the student: 

Even though you indicate that you are not having any 
particular problems in your subject areas, it would 
seem that, on the basis of your standard test results, 
you are not doing as well as might be expected in 
Course _• 



Store for write-out to counselor: 

The student indicates no problems in his subject areas, 
but does not seem to be doing as well as expectancy 
would predict in Cours e « 



III. Nature of the Problem 



-- write out reaction to messages to student using this form: 
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You indicate that you are having problems in 

and . Please type in the nature of your problem or 

your problems. A few words will be o.k. You don't have to go 
into much detail. If you cannot describe the problems you are 
having, please push the "R" key. 

--If student punches "R" key set flag for 1 to 1 referral. 

-- After student's response has been entered, write out to 
the student: 

Would you like to continue with this interview at this time, 
or would you like to discuss your problem with your 
counselor before you continue? 

A. I'll continue. 

B. I'd like to talk with a counselor first. 

-- If A 

Proceed to Section IV Post High School Plans 
-- If B 

Proceed to closing statement Section VIII Close 
Interview 

IV. Post High School Plans 

Do you plan to continue your education after high school? 

A. Yes 

B. No 



If A. -- write out: 

What kind of school do you plan to attend? 

If you are undecided at this time, press the "F" key and 
then type in the letters for the group you would most 
likely select if you had to choose now. 

A. Group A. -- Ivy League Colleges 

B. Group B. -- "Other" colleges and universities 
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C. Group C. -- California State Colleges 

D. Group D. -- Junior Colleges , ^ ^ , 

E. Group E. -- Technical and Vocational Schools 

F. I have not yet decided on the kind of school that I 

wish to attend. 

If "F" depressed -- write out to counselor: 

student indicates indecision in college or school 
choice at this time. 

-- record Group 

-- get S sex 

-- get S GPA 

-- search Expectancy Table I (see appendix) 

-- get projected high school grades 



-- write out: 



The grades you get in high school largely determine 
the type of college you will be able to attend. Let's 
just take a look at your projected high school grades 
based upon your present performance. 

Of students going on to Cubberley who have earned 
grades like you’rs in junior high: 

% earned A 
% earned B 
% earned C+ 

% earned C 
% earned D or F 

-- using the projected grades search Expectancy 
Table II (see appendix) for college groups the 
student has indicated above. 

-- is student's choice group removed from his optimal 
expectancy group? 
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-- store + or - 

if yes -- write out to student: 

On the basis of your projected grades your present 
college choice indicates that: 

(Choose appropriate lead in) 

+ = you are shooting pretty high. 

- = you are shooting pretty low. 

The fact that you are thinking about what you are 
going to do after high school is a good sign. The 
important thing is to have some ideas and be 
thinking about them, and what you might eventually 
do. You ought to acquaint yourself with college 
entrance requirements at the schools you consider. 

-- write out to counselor: 

This student's college choice indicates that he is not 
properly evaluating his abilities. (Choose the 
appropriate response) 

+ = This student's level of aspiration is 1 group above 
expectancy in his choice of colleges. 

- = This student is choosing colleges 1 group below 
what might be expected. 

If B. (referring to first question under Section IV, 

"Do you plan. . .") 

-- write out: 

What do you plan to do after high school? 

A. Get a job 

B. Join the military 

C. I have not yet decided 

D. If the above choices do not include what you 
intend to do after high school, please push the 
"D" key and type in your plans. 

E. Become a housewife 
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If C - - write out to student: 

You don't have to be fixed on a specific goal 
right new, but the important thing is to have 
some ideas and be thinking about what you 
might eventually do. Don't just allow your- 
self to drift into something, but go someplace 
that you want to go. 

-- get S sex 
- - get S GPA 

-- search Expectancy Table I (see appendix) 

-- get projected high school grades 

-- using projected grades, search Expectancy Table II. 

-- does the student's optimal expectancy lie below college 
levels ? 

If no -- write out to counselor: 

This student does not appear to be basing his post 
high school plans on his projected academic ability. 
Other considerations ought to be looked at, and 
encouragement sought and offered. 

At present which of the following areas seems to appeal to you 
most as a field for further education or employment? 

A. Business 

B. Physical Science 

C. Biological Science 

D. Social Science 

E. Language 

F. Music and Arts 

G. Technical or Vocational Fields 

Now that we've got some idea of what it is that you'd like to do 
after high school, let's look at your high school course plans. 



20 August 1965 



172 



TM-2611/000/00 



V. Course Plans 



The required courses for the 10th grade are: 

English 2 

Contemporary World History and Geography 
Physical Education 

In addition to these courses please enter the Code Numbers of the 
other courses that you would like to take. (For the courses avail- 
able and their Code Numbers, please see page of the Informa- 
tional pamphlet. A work sheet to help you in keeping a record of 
your course selections is provided on page of this pamphlet. ) 

If for some reason you find that you are unable to make your course 
selections at this time, please press the "R" key. 

-- if "R" key depressed, proceed to closing statement Section VIII 
Close Interview. 

machine check on number of solids 

Is the number of solids 4 ? 

If yes -- check S average -- average B? 

If no -- write out to student: 

In order to carry five solids you. should have an overall "B" 
average in your previous year's course work. 

Present following only for 10th grade: 

Keeping the course load to four solids helps you to do better 
in your academic subjects and also gives you a chance to 
explore non-academic areas. 

If it is desirable for your planning to take additional courses, 
or to get courses out of the way so you can take other 
courses in which you have a special interest, you might 
consider taking some summer courses. 

To counselor write out: 

On the basis of course plans for the 10th grade, it would 
be well to earmark this student for a counseling consultation 
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early in the 10th year, so that his course load can be 
re-evaluated and lightened if desirable or if necessary. 

Are you satisfied with the courses you have indicated, or would 
you like the opportunity to change them some way ? 

A. I'm satisfied. 

B. I would like to change them. 

If B - - print out elected courses with this lead line: 

Type in the letter of the course(s) you would like to change 
and the Code Number of the course you would like to 
choose in its place. 

Do you anticipate taking any courses during the summer ? 

A. Yes 

B. No 

If A -- write out: 

Type in the Code Number(s) of the course(s) you plan to 
tske. 

Present only after the 10th grade: 

At the ninth grade your projected 11th and 12th grade 
schedule is considered to be tentative. Firm scheduling 
decisions can be made only on the basis of your year by 
year performance. What you decide to take in the 11th 
and 12th grade will very much be governed by what it is 
that you want to do when you get out of high school. 

Since a person's plans tend to change over a period of 
time, you should see your counselor toward the end of 
your 10th year in order to evaluate your 11th and 12th 
grade program in the light of your current goals. 

Although we will be talking only about tentative plans, 
which you will be able to change, let's look at your high 
school course plans as you see them now. 
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Now let's look at your 11th grade schedule. 

The required courses for the 11th grade are: 

U. S. History 
English 3 

Physical Education 

In addition to these courses, please enter the Code Numbers of the 
other courses that you would like to take. 

machine check on solids 

number greater than 4 ? 

If yes -- average B? 

If no -- present only the first part of the message presented 
above . 

In order to take five solids, you should have an overall 
B average in your previous year's work. 

Repeat pages 7 and 8 

Are you satisfied with the courses you have indicated, or 
would you like the opportunity to change them some way ? 

A. I'm satisfied. 

B. I would like to change them. 

If B -- print out elected courses with this lead line: 

Type in the letter of the course(s) you would 
like to change and the Code Number of the course 
you would like to choose in its place. 

Do you anticipate taking any courses during the summer ? 

A. Yes 

B. No 

If B -- write out: 

Type in the Code Number(s) of the course(s) 
you plan to take. 
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Now let's look at your 12th grade schedule. 

The required courses for the 12th grade are: 

Government II 
Physical Education 

In addition to these courses please enter the Code Numbers of the 
other courses which you would like to take. 

machine check on solids 

number greater than 4 ? 

If yes -- average B? 

If no -- present only the first part of the message 
presented above. 

In order to take five solids, you should have an overall 
B average in your previous year's work. 

Repeat pages 7 and 8 

Are you satisfied with the courses you have indicated, or 
would you like the opportunity to change them some way ? 

A. I'm satisfied. 

B. I would like to change them. 

If B -- print out elected courses with this lead line: 

Type in the letter of the course(s) you would like 
to change and the Code Number of the course you 
would like to choose in its place. 

Do you anticipate taking any courses during the summer? 

A. Yes 

B. No 

If B -- write out: 

Type in the Code Number(s) of the course(s) you 
plan to take. 
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machine check: 

Is the student college bound? 

Check the answer to question about further schooling and 
kind of post high school plans. 

If yes: 

Does the student's proposed schedule contain 
sufficient special units to correspond with the kind 
of college or school that he wishes to attend? 

Check his answers to questions in post high school plans. 

Group A = 7 or more special units 
Group B = 3-6 special units. 

Group C = 0-2 special units 

Group D = no special units are required 

If yes -- go on 

If no -- store for future write out to student: 

In order to attend a college or university in Group , 

you will have to revise your high school course plans 
to include more special units. You have chosen, as 

your plans stand now, special units. For 

Group schools, you need more special units. 

If the student has chosen Group A or B add: 

However, your present course plans may stand if you 
Wish to change your choice of colleges. 

Write out to counselor: 

The student seems to need some assistance in planning 
courses for the 11th and 12th grade. Some explanation 
of college requirements in terms of special units is 
in order. 

machine -- 

Does this projected schedule, including summer courses, 
contain courses relevant to his chosen area of 
education or employment? 
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Compare student's projected schedule with area in 
post high school plans. 

What is the indicated major area? 



If it is: Then it should include: 



A. 


(Business) 


4 years 


English 


B. 


(Physical Science) 


II 


" Math IV Physics 


C. 


(Biological Science) 


II 


" Biology Chemistry 


D. 


(Social Science) 


II 


It 


E. 


(Language) 


It 


" 2 or more years 

of at least 1 
foreign language 


F. 


(Music and Arts) 


II 


II 


G. 


( Technical/ Vocational) 


II 


II 



Using an exception type report on students who do not have the 
courses in the above table included in their schedule, store for 
future write- out to student these rules: 

If in categories A-F not taking 4 years English: 

It is a good idea for all college bound students to take 
four years of English. 

If in category B not taking Math IV or Physics: 

Students planning to go into the physical sciences would 
do well to include Math IV and Physics in their course 
program. 

If in category C not taking Biology or Chemistry: 

Students planning to go into the biological sciences would 
do well to include Biology and Chemistry in their course 
program. 

If in category E not taking 2 or more years of at least 1 
foreign language: 

Students planning to go into languages should include 2 or 
more years of at least 1 foreign language in their course 
program. 
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Write out accumulated messages. 

Well, that about covers our examinations of your projected high 
school plans. Would you check over this print- out of your courses ? 

-- print out S course plans 

How do you feel about this schedule ? 

A. I want my course plans to stay as they are. 

B. I want my coursj plans to stay as they are, although I 
have some reservations about them. 

C. I would like to revise my course plans. 

D. I would like to confer with a counselor about my course plans. 
If A -- record for counselor print out: 

This student wants course plans to remain just as they are. 

If B - - record for counselor print out: 

This student has some reservations about his course plans. 

If C -- print out previous course plans for grades 10, 11, 
and 12. 

repeat choice cycle 

Type in the letter of the course(s) you would like to 
change and the Code Number of the course you would 
like to choose in its place. 

If D -- set flag for counselor referral. 

Do you have any further concerns or any questions about any 
information that has been given to you ? 

A. Yes 

B. No 

If A - - write out: 

Please feel free to type in your questions or concerns 
and they will be referred to your counselor. 



o 
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If B - - proceed to final information print- out (counselor 
expectations, high school expectations, etc. ) 

VI. High School Expectations 

--Informational rules used by Hutch to be given to all students 
reaching this point in the interview. Write out: 

You might be wondering about how things are done over at 
Cubberley. 

The first event on the first day will be a class meeting (place is 
to be posted on bulletin boards). Here you will meet your class 
and school officials. Following this meeting, you will be given 
a student handbook, general information bulletin, and a class 
schedule. From here on out you will attend shortened revisions 
of your regular classes. In case you are confused about the 
terms used at Cubberley, their lanes are the same as your 
tracks, and their blocks are the same as yo. r periods. The 
blocks are called by letter names A, B, C, D rather than by 
numbers 1, 2, 3, 4. 

VII. Counselor Expectations 
-- write out: 

When you get up to the high school it would be well if you would 
get to know your counselor as soon as possible. He*s there to 
help you and he'll be writing recommendations when it comes 
time to think about jobs or college. So it's to your advantage to 
make the first move in getting to know him. 

VIII . Close Interview 
--write out: 

We hope this interview has provided you with some information 
to think about in planning your high school program. 

It's been nice interacting with you. 

The interview is now ended. Thank you for your cooperation. 



4 



20 August 1965 



180 



TM-2611/000/00 



APPENDIX B 

PRE- INTER VIEW PUPIL APPRAISAL 
COUNSELOR 

Analysis of Cumulative Folder 



The statements concerning the student's projected high school GPA 
and the type of college he could enter should be based not only on 
actuarial predictions but also on any additional information about the 
student which would modify appraisal statements. If you feel the 
actuarial predictions do not apply to this student, please modify these 
on the basis of your judgment. 



Grade Point Average 

Place the number of chances out of ten the student has for 
achieving the following grade ranges in the blanks provided. 



out of ten less than 2. 00 (D and F) 



out of ten 2.00 to 2. 49 (C) 



out of ten 2.50 to 2.99 (C+) 



out of ten 3.00 to 3.49 (B) 



out of ten 3. 50 to 4. 00 (A) 



If there are any comments you wish to make please do so on the back 
of this form. 



Name of Student 
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APPENDIX C 



PUPIL APPRAISAL CHECK LIST 

Instructions to Counselor. Study the data in the cumulative for each 

student assigned in this experiment. After studying each folder place 

a check before the statements that apply to that student. Place any 

additional comments at the bottom or on the back of this form. 

Student's Name 

1. Student's grades have gond down quite a bit. Ask about this 

in interview. Possibly there are personal problems. 

2. Student's grades have gone up. Ask about this at the next 

interview. 

3. This student should be watched closely. He will probably 

need remedial courses. 

4. Student is a potential drop-out. 

5. Should be headed for college. Encourage student to explore 

widely in academic areas. 

6. Low counseling priority. No problems apparent. 

7. Student is getting better grades than one would expect from 

looking at aptitude scores. 

8. Student is not achieving* as well as aptitude scores would 

predict. 

9. Cumulative folder contains no aptitude scores for this 

student. Try to get sonle. 

10. Look out for over- ambitious plans. 

Student may need to strengthen quantitative skills or will 

probably experience academic difficulty. 

12. Student should improve verbal skills. If not, student may 

not be able to attain desired academic goals. 

13. The disparity between aptitude scores and achievement is so 

great that one is led to suspect aptitude test. Check test. 

14. Cumulative folder contains no grades for this student. 
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APPENDIX D 

POST INTERVIEW PUPIL APPRAISAL 
COUNSELOR 

Analysis of Interview 



The answers to the following questions need not be based on actuarial 
predictions alone. Your responses may be modified by any additional 
information you wish to use. 

1. Does the student indicate he is having problems with his courses ? 
yes no 

Z. Does the student deny any problems with courses when according 
to the expectancy scores he is having problems? yes no 

3. Does the student indicate he is going to college ? yes no 

4. Is he realistic in his plans to attend college ? yes no 

(Guide to Question 4. If he is planning on college and does not have 
at least a .50% chance of entering some type of college or trade 
school, fo.r example a 40% chance of entering a trade school and a 
10% chance of entering a junior college, he is not being realistic. 

If he is not planning on college but has a 50% chance of entering 
some type of college or trade school he is not being realistic. ) 

5. Is he planning to take more solids than his GPA warrants ? 

yes no 

6. Is he planning to take enough special units to enter the type of 

college he has selected? yes no 

(If he is not planning to enter college the answer would be yes. ) 

7. Is he planning on taking the courses necessary to prepare him for 

his proposed college major? yes no 

(If he is not planning on college the answer would be yes. ) 

Name of Student 



Name of Counselor 
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APPENDIX E 



STUDENT’S COURSE SELECTION FORM 



Please place the title of the courses selected for the 10th grade in the 
space below. Be specific. For example, write in Art I, not just Art. 
Also include any summer courses to be taken between the 9th and 10th 
grades and indicate these are summer courses by placing (S) behind 

the course title. 



World History and Geography 




English II 




Phvsical Education 









Do you have any reservations about your 10th grade courses 

' yes or no 

Please place the title of the courses selected for the 11th 

grade in the spaces below. Also include any summer courses to be 

taken between the 10th and 11th grade. Mark these with (S). 

U. S. History — 

English III - 

Physical Education 



Do you have any reservations about your 11th grade courses ?_ 

' yes or no 

Please place the title of the courses selected for the 12th 

grade in the spaces below. Also include any summer courses to be 

taken between the 11th and i2th grade. (Mark these with (S). 

Government II 

Physical Education - 



reservations about your 12th grade courses? 



Do you have any 



ves or no 
# 






WBSSssm 
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APPENDIX F 



QUESTIONS FOR ANALYSIS OF MACHINE LIMITATIONS 



Questions to be asked of those students who did not develop total 
course plans during the machine interview: 

1, Was your inability to make total course plans caused by the 
machine not presenting the material clearly? 

2. Did you feel the machine did not give you enough information to 
make adequate choices? If not, in what areas do you feel more 
information is needed? 



Questions to be asked of all students: 

1, Did the machine interview bore you or cause you to become 
restless ? 

2, Did the machine, by not letting you know if your responses were 
good or poor, make it difficult to make choices ? 

3, Did the lack of a counselor being in the room while you were 
having the machine interview make it more difficult to make 
decisions ? 

4, Did the automated interview meet your needs in terms of high 
school course planning? 

5, Did the automated interview meet your expectations of what an 
educational planning interview should be ? 
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APPENDIX G 

AREAS TO BE COVERED IN COUNSELOR'S INTERVIEW 



In addition to the areas usually discussed in your educational planning 
interviews, please include the following items. 

1. What courses is the student currently taking? 

2. Does the student indicate he is having any problems with his 
present courses ? 

3. Does the student intend to enter college? 

4. If so, does he plan to enter 

a. Ivy League Colleges, 

b. "other" colleges and universities, 

c. California State Colleges, 

d. junior colleges, or 

e. technical or vocational schools? 

5. Have the student make course plans for the 10th grade. 

6. Have the student make course plans for the 11th grade. 

7. Have the student make course plans for the 12th grade. 

8. If the student is college bound have him select a tentative college 

majoi from the list below. 

a. Business 

b. Physical Science 

c. Biological Science 

d. Social Science 

e. Language 

f. Music and Art 

g. Technical and Vocational 

9. If, at the close of the interview, the student wishes to change any 
courses he has previously selected for high school, permit him 
to do so. 
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APPENDIX H 



ACTUAL AUTOMATED INTERVIEW 



YOU ARE ABOUT TO PARTICIPATE IN AN INTERVIEW WITH AN 
AUTOMATED COUNSELOR. 

THE COMPUTER WILL HELP YOU PLAN WHAT YOU WANT TO DO 
IN HIGH SCHOOL AND IN LATER LIFE. 

BUT FIRST, JUST TO LET THE COMPUTER KNOW YOU ARE THERE 
PLEASE TYPE THE LETTER A AND A CARRIAGE RETURN 
(RED KEY ON THE RIGHT- H/.ND SIDE OF THE KEYBOARD). 

jjcjjcjjcsje A 

GOOD. . . 

THE MOST IMPORTANT KEY IS THE CR KEY (RED KEY ON THE 
RIGHT). YOU WILL USE IT TO LET THE COMPUTER KNOW THAT 
YOU HAVE MADE AN ANSWER. JUST FOR PRACTICE, TYPE THE 
LETTER B AND A CR (RED KEY). 

j5t>!<5!o;e B 

OK 

SOMETIMES YOU WILL BE ASKED TO ANSWER A YES OR NO 
QUESTION. WHEN YOU ARE, JUST TYPE YES OR NO AND A CR 
DON'T FORGET THE CR. 

LET'S TRY IT. HERE IS A QUESTION--- 
ARE YOU IN CALIFORNIA NOW ? 

PLEASE TYPE YES OR NO. 

YES 

FINE. . 

MANY OTHER PEOPLE ARE USING THIS COMPUTER RIGHT NOW. 
ONCE IN A WHILE THE COMPUTER WILL HAVE TO TAKE CARE OF 
THEIR URGENT NEEDS. WHEN THIS HAPPENS, YOU MAY FIND 
THAT YOU HAVE TO WAIT A MOMENT OR TWO. DON'T WORRY, 
THE COMPUTER WILL COME BACK TO TAKE CARE OF YOU AS 
SOON AS IT CAN. 

WHEN YOU HAVE READ THIS, TYPE OK AND A CR. 
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^JcJcjjcic OK 

GOOD. 

BY NOW YOU HAVE NOTICED THAT BEFORE YOU TYPE AN 
ANSWER, THE TYPEWRITER TYPES STARS LIKE THIS- 

❖❖❖❖ 

THIS IS TO LET YOU KNOW THAT IT IS TIME FOR YOU TO TYPE 
SOMETHING. YOU SHOULD NEVER TYPE ANYTHING UNTIL AFTER 
THE STARS ARE TYPED OUT BY THE COMPUTER. 

DO YOU UNDERSTAND? 

PLEASE TYPE YES OR NO. 

YES 

ANOTHER VERY IMPORTANT KEY IS THE / OR "SLASH" KEY. 

IT IS THE GREEN KEY ON THE BOTTOM RIGHT OF THE KEYBOARD. 
LOOK FOR THE / KEY (GREEN KEY). WHEN YOU FIND IT, TYPE 
/ AND A CR. 

SOMETIMES YOU WILL HAVE TO TYPE MORE THAN ONE LINE AND 
THE COMPUTER CANNOT TELL IN ADVANCE HOW MANY LINES 
YOU WILL TYPE. IN SUCH A CASE, YOU WILL LET THE COMPUTER 
KNOW THAT YOU HAVE FINISHED BY TYPING A / AND A CR. 

WHAT YOU TYPE WILL LOOK SOMETHING LIKE THIS 

1 

❖ / 

LET'S TRY IT. 

TYPE 1 AND A CR, 2 AND A CR, AND A / AND A CR. 

(WAIT FOR THE STARS. ) 

❖❖❖❖ 1 
Sjcsjc## 2 

❖❖❖❖ / 



FINE. . 
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PLEASE TYPE YOUFL ID NUMBER AND A CR 
61 

IS YOUR NAME JANE DOE 

YES 

ACCORDING TO MY RECORDS THIS IS HOW YOU DID LAST SEMESTER. 



CODE COURSE GRADE 

1 P.E. B 

2 ENGLISH I C 

3 GENERAL SCIENCE C 

4 EUROPEAN HISTORY B 

5 TYPING B 

6 GENERAL MATH C 



IS THIS CORRECT? IF SO, TYPE YES IF NOT, TYPE NO 

YES 

ARE YOU HAVING ANY PROBLEMS WITH ANY OF THESE COURSES? 

PLEASE TYPE YES OR NO.. 

NO 

OK. 

DO* YOU PLAN TO CONTINUE YOUR FORMAL EDUCATION BEYOND 
HIGH SCHOOL ? 

YES , - „ T^T A^T 

OF THE FOLLOWING IS MOST LIKE THE SCHOOL lOU PLA 

TO ATTEND? 

1 IVY LEAGUE COLLEGE 

2 OTHER COLLEGE OR UNIVERSITY 

3 CALIFORNIA STATE COLLEGE 

4 JUNIOR COLLEGE 

5 BUSINESS OR VOCATIONAL SCHOOL 

6 I HAVE NOT YET DECIDED 
TYPE THE APPROPRIATE CODE. 

4 

DO YOU INTEND TO MAJOR IN ENGINEERING OR ONE OF THE 
SCIENCES ? 

PLEASE TYPE YES OR NO. 



me 



5jC*5lCJlC NO 
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WHICH OF THE FOLLOWING APPEALS TO YOU MOST AS A COLLEGE 
MAJOR--- 

1 BUSINESS 

2 SOCIAL SCIENCE 

3 MUSIC AND ARTS 

4 FOREIGN LANGUAGES 

5 EDUCATION 

6 ENGLISH 

TYPE THE APPROPRIATE CODE. 

OK 

THE GRADES YOU GET IN HIGH SCHOOL LARGELY DETERMINE 
THE TYPE OF COLLEGE YOU WILL BE ABLE TO ATTEND. 

LET'S JUST TAKE A LOOK AT YOUR PROJECTED HIGH SCHOOL 
GRADES BASED ON YOUR PRESENT PERFORMANCE. 

STUDENTS WHO GOT GRADES LIKE YOURS IN JUNIOR HIGH, 

DID AS FOLLOWS AT CUBBERLEY--- 

10 PER CENT EARNED A OR B 
60 PER CENT EARNED C+ 

30 PER CENT EARNED C 

ON THE BASIS OF YOUR PROJECTED GRADES, YOUR PRESENT 
COLLEGE CHOICE INDICATES THAT YOU ARE THINKING PRETTY 
REALISTICALLY ABOUT YOUR FUTURE LIFE. 

NOW THAT WE'VE GOT SOME IDEA OF WHAT YOU'D LIKE TO DO 
AFTER HIGH SCHOOL, LET'S LOOK AT YOUR HIGH SCHOOL 
COURSE PLANS. IN THE lOTH GRADE, THE REQUIRED COURSES 
ARE--- 

ENGLISH 2 

CONTEMPORARY WORLD HISTORY AND GEOGRAPHY 
PHYSICAL EDUCATION 

WHAT OTHER COURSES WOULD YOU LIKE TO TAKE NEXT YEAR? 
LOOK AT THE LIST OF COURSES. 

WHEN YOU HAVE DECIDED WHAT COURSES YOU WANT TO TAKE, 
TYPE YES. 

IF YOU CAN'T MAKE UP YOUR MIND AT ALL TYPE NO 
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YES 

GOOD. 

TYPE THE CODE NUMBER OF EACH COURSE YOU WANT TO TAKE. 
TYPE A CR AFTER EACH NUMBER. 

WHEN YOU ARE ALL THROUGH, TYPE A / AND A CR. 

24 

16 

50 

OK, HERE IS THE PROGRAM YOU HAVE PROPOSED 
ENGLISH 2 

CONTEMPORARY WORLD HISTORY AND GEOGRAPHY 
PHYSICAL EDUCATION 

TYPING 11 S 
GENERAL BUSINESS S 
FOODS 1 

ARE YOU SATISFIED WITH THIS PROGRAM? 

IF SO, TYPE YES 

IF YOU WOULD LIKE TO CHANGE IT, TYPE CHANGE 
YES 

OK 

THAT LOOKS LIKE A PRETTY GOOD lOTH GRADE PROGRAM. 

DO YOU ANTICIPATE TAKING ANY SUMMER COURSES THAT 
YEAR? 

YES 

GOOD. TYPE THE CODES OF THOSE COURSES YOU WILL TAKE 
IN THE SUMMER, 

TYPE A CR AFTER EACH NUMBER. 

WHEN YOU ARE ALL THROUGH, TYPE A / AND A CR. 
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that means that in summer school you will 

TAKE DRIVER EDUCATION 
OK • 

At'tHE 9TH GRADE YOUR PROJECTED IITH AND 12TH GRADE 
SCHEDULE IS CONSIDERED TO BE TENTATIVE. FIRM DECISIONS 
CAN BE MADE ONLY ON THE BASIS OF YOUR YEAR BY YEAR 
PERFORMANCE. 

WHAT YOU DECIDE TO TAKE IN THE IITH AND 12TH GRADES 
WILL BE GOVERNED LARGELY BY WHAT IT IS YOU WANT TO DO 
AFTER HIGH SCHOOL. SINCE A PERSON'S PLANS TEND TO CHANGE 
OVER A PERIOD OF TIME, YOU SHOULD SEE YOUR COUNSELOR 
TOWARD THE END OF YOUR lOTH YEAR IN ORDER TO EVALUATE 
YOUR IITH AND 12TH GRADE PROGRAM IN THE LIGHT OF YOUR 
CURRENT GOALS. 

ALTHOUGH WE WILL BE TALKING ABOUT TENTATIVE PLANS, 
WHICH YOU WILL BE ABLE TO CHANGE, LET'S LOOK AT YOUR 
HIGH SCHOOL COURSE PLANS AS YOU SEE THEM NOW. 

LLT'S THINK ABOUT THE IITH GRADE. 

THE REQUIRED COURSES FOR THE IITH GRADE ARE--- 

ENGLISH 3 
U.S. HISTORY 
P.E. 

WHAT OTHER COU ISES WOULD YOU LIKE TO TAKE IN THE IITH 
GRADE? LOOK AT THE LIST OF COURSES. 



WHEN YOU HAVE DECIDED WHAT COURSES YOU WANT TO TAKE. 
TYPE YES. IF YOU CAN'T MAKE UP YOUR MIND AT ALL TYPE 

NO 



YES 

GOOD. „ 

type the code NUMBER OF EACH COURSE YOU WANT TO TAKE. 

TYPE A CR AFTER EACH NUMBER. 

WHEN YOU ARE ALL THROUGH. TYPE A / AND A CR. 



18 






mmmm 
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WHAT? TRY AGAIN 

#### 18 

29 

#s!<s!<s!e 93 

sjc^jsjSe# / 

SOMETHING IS WRONG. YOU SEEM TO HAVE PUT IN SOME 
WRONG NUMBERS. START YOUR LIST FROM THE BEGINNING 
AGAIN, PLEASE. 

TYPE THE CODE NUMBER OF EACH COURSE YOU WANT TO TAKE. 
TYPE A CR AFTER EACH NUMBER. 

WHEN YOU ARE ALL THROUGH, TYPE A / AND A CR. 

#### 18 



29 

>!«>!« sis jjc 93 

sjcsies^rsis / 

OK, HERE IS THE PROGRAM YOU HAVE PROPOSED 

ENGLISH 3 
U. S. HISTORY 
P. E. 

CLERICAL PRACTICE S 
TYPING 22 S 
MATH M 1 

ARE YOU SATISFIED WITH THIS PROGRAM? 

IF SO, TYPE YES 

IF YOU WOULD LIKE TO CHANGE IT, TYPE CHANGE 

SjS sic sic sic YES 

OK 

THAT LOOKS LIKE A REASONABLE IITH GRADE PROGRAM. 

DO YOU ANTICIPATE TAKING ANY SUMMER COURSES THAT YEAR? 
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NO 

NOW LET'S THINK ABOUT THE 12TH GRADE. 

THE REQUIRED COURSES FOR THE 12TH GRADE ARE--- 



GOyERNMENT 2 
PHYSICAL EDUCATION 



(ENGLISH 4 IS NOT REQUIRED. BUT YOU MAY WANT TO TAKE IT. ) 

WHAT OTHER COURSES WOULD YOU LIKE TO TAKE IN THE 12TH 
GRADE? LOOK AT THE LIST OF COURSES. 

WHEN YOU HAVE DECIDED WHAT COURSES YOU WANT TO TAKE. 
type yes. if you CAN'T MAKE UP YOUR MIND AT ALL TYPE 

NO 



YES 

TYPE THE CODE NUMBER OF EACH COURSE YOU WANT TO TAKE. 
TYPE A CR AFTER EACH NUMBER. 

WHEN YOU ARE ALL THROUGH. TYPE A / AND A CR. 



51 

15 

92 

j 

OK. HERE IS THE PROGRAM YOU HAVE PROPOSED 

GOVERNMENT 2 
PHYSICAL EDUCATION 



FOODS 2 
BOOKKEEPING 1 
MATH REVIEW 



ARE YOU SATISFIED WITH THIS PROGRAM? 

IF SO, TYPE YES 

IF YOU WOULD LIKE TO CHANGE IT. TYPE CHANGE 



YES 
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t 



h 






OK 

THAT'S A PRETTY GOOD 12TH GRADE PROGRAM. 

DO YOU ANTICIPATE TAKING ANY SUMMER COURSES THAT YEAR? 

NO 

IT IS A GOOD IDEA FOR ALL COLLEGE BOUND STUDENTS TO TAKE 
4 YEARS OF ENGLISH. GIVE THIS SOME THOUGHT. 



I HOPE THIS INTERVIEW HAS PROVIDED YOU WITH SOME INFOR- 
MATION TO THINK ABOUT IN PLANNING YOUR HIGH SCHOOL 
PROGRAM. 

IT'S BEEN NICE INTERACTING WITH YOU. 

THANK YOU. JANE 

THIS IS THE END OF THE INTERVIEW 






f 



